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Abstract. A wide class of problems in combinatorics, computer science and physics can be described along the fol¬ 
lowing lines. There are a large number of variables ranging over a finite domain that interact through constraints that 
each bind a few variables and either encourage or discourage certain value combinations. Examples include the A:-SAT 
problem or the Ising model. Such models naturally induce a Gibbs measure on the set of assignments, which is charac¬ 
terised by its partition function. The present paper deals with the partition function of problems where the interactions 
between variables and constraints are induced by a sparse random (hyper)graph. According to physics predictions, a 
generic recipe called the “replica symmetric cavity method” yields the correct value of the partition function if the under¬ 
lying model enjoys certain properties [Krzkala et al., PNAS 2007]. Guided by this conjecture, we prove general sufficient 
conditions for the success of the cavity method. The proofs are based on a “regularity lemma” for probability measures 
on sets of the form O” for a finite il and a large n that may be of independent interest. 
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1. Introduction 

Despite their simplicity, or perhaps because thereof, the first and the second moment method are the most widely 
used techniques in probabilistic combinatorics. Erdos employed the first moment method famously to lower- 
bound the Ramsey number as well as to establish the existence of graphs of high girth and high chromatic num¬ 
ber j24j[25|. Even a half-century on, deterministic constructions cannot hold a candle to these probabilistic re¬ 
sults jT2][35). Moreover, the second moment method has been used to count prime factors [42j and Hamilton 
cycles |38j as well as to determine the two possible values of the chromatic number of a sparse random graph (3j . 

Yet there are quite a few problems for which the standard first and the second moment methods are too simplis¬ 
tic. The random k-SAT model is a case in point. There are n Boolean variables x\,...,x n and m clauses ,a. m , 
where m = \ari\ for some fixed a > 0. Each clause binds k variables, which are chosen independently and uni¬ 
formly, and discourages them from taking precisely one of the 2 k possible truth value combinations. The forbidden 
combination is chosen uniformly and independently for each clause. 

The random /c-SAT instance O = ®fc(n, m) gives rise to a probability measure on the set {0, l}' 1 of all Boolean 
assignments naturally. Indeed, for a given parameter yS > 0 the Gibbs measure is defined by letting 

2 m 

p® /3 (o 0 =-P[ exp(-/il {a violates ap) for every assignment cr e {0,1}”, where (1.1) 

i= i 

m 

Zp(<S>)~ Y ]~[ exp(-/fl {cr violates ap) (1.2) 

(7£{0,l} n i= 1 

is called the partition function. Thus, the Gibbs measure weighs assignments according to the number of clauses 
that they violate. In effect, by tuning p we can interpolate between just the uniform distribution on {0,1}" {p - 0) 
and a measure that strongly favours satisfying assignments (/f —>• oo). Hence, if we think of <D as inducing a “height 
function” a >-► #{clauses of fl> violated by a} on the set of assignments, then varying /) allows us to explore the re¬ 
sulting landscape. Apart from its intrinsic combinatorial interest, the shape of the height function, the so-called 
“Hamiltonian”, governs the performance of algorithms such as the Metropolis process or Simulated Annealing. 

To understand the Gibbs measure it is key to get a handle on the partition function Zp{ O). Of course, the 
default approach to this kind of problem would be to apply the first and second moment methods. However, upon 

* A preliminary version [3] of this paper, presented by the first author at RANDOM 2015 and by the seocnd author at the RS&A 2015 confer- 
ence, contained a critical technical error that affected its main results. This present version is based on similar key insights but the main results 
are different from the ones stated in [8]. 

* * The research leading to these results has received funding from the European Research Council under the European Union’s Seventh 
Framework Programme (FP/2007-2013) / ERC Grant Agreement n. 278857-PTCC. 
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closer inspection it emerges that Zp(<D) < exp(-0(n))E[Z^(®)] with high probability for any a,f’> > 0 (5). In other 
words, the first moment over-estimates the partition function of a typical random formula by an exponential factor. 
The reason for this is a “lottery effect”: a tiny minority of formulas render an exceptionally high contribution to 
E[Z^(®)]. Unsurprisingly, going to the second moment only exacerbates the problem and thus for any a,p > 0 we 
find E[Zp{<!>) 2 ] > exp(Q( n) ) E [Zjg (®) ] 2 . In other words, the second moment method fails rather spectacularly for all 
possible parameter combinations. 

The first and the second moment method fall victim to similar large deviations effects in many alike "random 
constraint satisfaction problems”. These problems, ubiquitous in combinatorics, information theory, computer 
science and physics l4l l3Tll37l . can be described along the following lines. A random factor graph, chosen either 
from a uniform distribution (like the random fc-SAT model above) or from a suitable configuration model, induces 
interactions between the variables and the constraints. The variables range over a fixed finite domain Q and each 
constraint binds a few variables. The constraints come with “weight functions” that either encourage or discourage 
certain value combinations of the incident variables. Multiplying up the weight functions of all the contraints just 
like in CED-dZ) , we obtain the Gibbs measure and the partition function. 

With the standard first and second moment method drawing a blank, we seem to be at a loss as far as calcu¬ 
lating the partition function is concerned. However, physicists have put forward an ingenious albeit non-rigorous 
alternative called the cavity method (31 j. This technique, which applies almost mechanically to any problem that 
can be described in the language of sparse random factor graphs, yields an explicit conjecture as to the value of 
the partition function. More specifically, the cavity method comes in several installments. In this paper, we are 
concerned with the simplest, so-called "replica symmetric” version. 

In one of their key papers (29) physicists hypothesized abstract conditions under which the replica symmetric 
cavity method yields the correct value of the partition function. The thrust of this paper is to prove corresponding 
rigorous results. Specifically, according to (29) the replica symmetric cavity method gives the correct answer if 
the Gibbs measure satisfies certain correlation decay properties. For example, the Gibbs uniqueness condition 
requires that under the Gibbs measure the value assigned to a variable x is asymptotically independent of the 
values assigned to the variables at a large distance from x in the factor graph. In Coro11 a rv l4.6l below we prove that 
this condition is indeed sufficient to guarantee the success of the cavity method. Additionally, Theorems l4.4l and l4.5l 
yield rigorous sufficient conditions in terms of substantially weaker conditions, namely a symmetry property and 
the non-reconstruction property. 

A key feature of the paper is that we establish these results not for specific examples but generically for a very 
wide class of factor graph models. Of course, stating and proving general results requires a degree of abstraction. In 
particular, we resort to the framework of local weak convergence of graph sequences (30] Part 4]. This framework 
suits the physics predictions well, which come in terms of the “limiting tree” that describes the local structure of a 
large random factor graph. To be precise, the replica symmetric prediction is given by a functional called the Bethe 
free energy applied to an (infinite) random tree. 

The principal tool to prove these results is a theorem about the structure of probability measures on sets of the 
form n" for some fixed finite set Q and a large integer n, Theo rem 12. 1 1 bel ow. We expect that this result, which is 
inspired by Szemeredi’s regularity lemma [40], will be of independent interest. To prove our results about random 
factor graphs, we combine Theorem l2.1l with the theory of local weak convergence to carry out completely gener¬ 
ically “smart” first and second moment arguments that avoid the lottery effects that the standard arguments fall 
victim to. 

In Section[2]we begin with the abstract results about probability measures on cubes. Subsequently, in Section[3] 
we set the stage by introducing the formalism of factor graphs and local weak convergence. Further, in Section[4] 
we state and prove the main results about Gibbs measures on random factor graphs. Finally, Section [5] contains 
the proof of a technical result that enables us to control the local structure of random factor graphs. 

Related work. A detailed (non-rigorous) discussion of the cavity method can be found in [31!. It is known that the 
replica symmetric version of the cavity method does not always yield the correct value of the partition function. For 
instance, in some factor graph models there occurs a “condensation phase transition” beyond which the replica 
symmetric prediction is off fl6ll29l . The more complex “1-step replica symmetry breaking (1RSB)” version of the 
cavity method (32) is expected to yield the correct value of the partition function some way beyond condensation. 
However, another phase transition called “full replica symmetry breaking” spells doom on even the 1RSB cavity 
method f5H . 
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The replica symmetric cavity method has been vindicated rigorously in various special cases. For instance, 
Montanari and Shah (33) proved that in the random /c-SAT model the replica symmetric prediction is correct up 
to the Gibbs uniqueness threshold. A similar result was obtained by Bandyopadhyay and Gamarnik (7] for graph 
colorings and independent sets. Furthermore, Dembo, Montanari and Sun [15) proved the replica symmetric con¬ 
jecture on a class of models with specific types of constraints. A strength of [193 is that the result applies even to 
sequences of non-random factor graphs under a local weak convergence assumption. But both [19] [33] are based 
on the “interpolation method” (26) [28] j36), which entails substantial restrictions on the types of models that can 
be handled. By contrast, the present proof method is based on a completely different approach centered around 
the abstract classification of measures on cubes that we present in Section[2] 

Since the “vanilla” second moment method fails on the random /c-SAT model, more sophisticated variants have 
been proposed. The basic idea is to apply the second moment method not to the partition function itself but to 
a tweaked random variable. For instance, Achlioptas and Moore (2) applied the second moment method to NAE- 
satisfying assignments, i.e., both the assignment and its binary inverse satisfy all clauses. Flowever, the number of 
NAE-satisfying assignments is exponentially smaller than the total number of satisfying assignments and thus this 
type of argument cannot yield the typical value of the partition function. The same is true of the more subtle ran¬ 
dom variable of Achlioptas and Peres (6)- Furthermore, the work of Ding, Sly and Sun (22) that yields the precise 
fc-SAT threshold for large k is based on applying the second moment method to a random variable whose con¬ 
struction is guided by the 1RSB cavity method. Among other things, the random variable from [22] incorporates 
conditioning on the local structure of the factor graph, an idea that will be fundamental to our arguments as well. 


Notation. If SC is a finite set, then we denote by 3A{SC) the set of probability measures on SC. Moreover, II • IItv 
signifies the total variation norm. If p is a probability measure on a product space SC 1 for finite sets SC, V and 
Sck, then pjs £ SA{.SC s ) denotes the marginal distribution of p on S. That is, if (x s ) se s £ SC s , then 

Mls(te)sES) = E M((*s)s£v)- 

(Xs) SE y\sE-9r v ' s 

If S - {v} for some veV, then we briefly write p [ „ rather than pj{ V }. 

The entropy of a probability measure p £ g?(3C) is denoted by H (p). Thus, with the convention that OlnO = 0 
we have iT(p) = - S-xaX pfx)lnp(x). Further, agreeing that Oln = 0 as well, we recall that the Kullback-Leibler 
divergence of p, v e FA {SC) is 


D(v||p)= V v(x)ln—— £ [0,oo], 

xTsc 

We are going to work with probability measures on sets Q" for a (small) finite Q and a large integer n a lot. If 
p £ PX(Cl n ), then we write op, ip for two independent samples from p. Where p is obvious from the context we just 
write or, r. Additionally, if X[a] is a random variable, then (X(a))^ — Y.aen n stands for the expectation of 

X with respect to p. Further, if a £ fi", 0^Sc|«] and coeCl, then we let 

o’MS] = |cr _1 MnS|/|S|. 

Thus, cr[-1S] is a probability distribution on Cl, namely the distribution of a(x) for a random x £ S. If S - {x} for 
some x £ [n], then we just write a[cj\x] rather than a[a)\{x}]. Clearly, <j[(i) |x] = l{cr(x) = «}. 

We use the (■)^ notation for averages over p £ PA (Cl") to avoid confusion with averages over other, additional 
random quantities, for which we reserve the common symbols E[- ], P[ - ]. Furthermore, we frequently work with 
conditional expectations. Elence, let us recall that for a probability space {3C,$4, P), a random variable X : SC —► IR 
and a cr-algebra PA c .<4 the conditional expectation C\X\.PX\ is a ^-measurable random variable on SC —* R such 
that for every ^-measurable event F we have E|1{F)E[ A|.7|| = E[1{F}X]. Moreover, recall that the conditional 
variance is defined as Var[X|^| = E[X 2 |^| - E[X|J?] 2 . 

In line with the two previous paragraphs, if Y : Q" —► IK is a random variable, p £ PC (Cl 11 ) and PF is a cr-algebra 
on C2 n , then we write (Y\PF) ^ for the conditional expectation, which is a ^-measurable random variable n £ Cl n « 
{Y\SC)^ (cr). Accordingly, for an event A c Cl" with p(A) > 0 we write (Y\A)^ - /p(A) £ IR for the expecta¬ 

tion of Y given A. 


3 



2. Probability measures on the cube 

In this section we present a general “regularity lemma” for probability measures on sets Q" for some finite set Q 
and a large integer n (Then re m 12. 1 1 bel ow). 


2.1. Examples. Needless to say, probability distributions on sets Q" for a small finite Q and a large integer n are 
ubiquitous. To get an idea of what we might hope to prove about them in general, let us look at a few examples. 

The simplest case certainly is a product measure /i = p®" with p e fifi(il). By the Chernoff bound, for any fixed 
£ > 0 there is no — no (£', O) > 0 such that for n > no we have 

(||<7[-[S] - pll^y) < £ for every Sc [n] such that |S| > en. (2.1) 

In words, if we fix a large enough set S of coordinates and then choose a randomly, then with probability close to 
one the empirical distribution on S will be close to p. 

As a twist on the previous example, let p e ./ZfiQ), assume that n is a square and define a measure p by letting 

y/n —1 

p(a>i,...,to„) = ]“[ 

1=0 


p^\+is/^ l ^i e : w j+is/n = w i+iVHl] • 


In words, the coordinates come in blocks of size y/n. While the values of all the coordinates in one block coin¬ 
cide and have distribution p, the coordinates in different blocks are independent. Although p is not a product 
distribution, 12.11 is satisfied for any fixed e > 0 and large enough n. Furthermore, if for a fixed k> 1 we choose 
jci, ..., x/c e [n] uniformly and independently, then 

E | Ph*!,...,**.) _ Ah*! ® •• • ® Vix k ||xv < £ < (2.2) 


provided that n> n\ (e, k, D) is sufficiently large. This is because for large enough n it is unlikely that two of the 
randomly chosen xi,...,Xk belong to the same block. 

As a third example, consider the set Q = {0,1} and the measure p defined by 


P ( 0 ) (W 1 ,...,W„) : 
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P (1) («!,...,«„) : 


E"=l “i / 1 \ n_ E" = i <»i 
2 


5 (M 


(0) 


+p (1) ). 


All the marginals pp, i e \n\, are equal to the uniform distribution on {0,1}. But of course the uniform distribution 
on Q” is a horrible approximation to p. Indeed, by the Chernoff bound with overwhelming probability a point 
drawn from p either satisfies - Lf = i ~ 1 /3 or E ( ! =1 a) ; ~ 2/3. However, the conditional distribution 

given, say, jj Z” =1 w i — 1/2, is close to a product measure. Thus, p induces a decomposition of O" into two “states” 
So = w i — 1/21, Si = {i£” =1 to,- > 1/2} such that p[- |So], p[- |Si] are close to product measures. 

As a final example, consider Q = {0,1}, assume that n is even and define p e 3?{Q. n ) by letting 


p(Wi ,...,&»„) 


I 1 \ ^~i>n!2 "i 


Y.i>n!2 tAl i 


In words, p is a product measure with marginal distribution Be(l/2) on the first n/2 coordinates and Be(l/3) on 
the other coordinates. Clearly, p satisfies 12.11 with p = Be(l/2) for sets S c [n/2] and with p = Be(l/3) for sets 
Sc[«]\ [n/2], provided that n is large. 

In summary, the following picture emerges. The conditions 12.11 and 12.21 are proxies for saying that a given 
measure p resembles a product measure. Furthermore, in order to obtain from a given p measures that satisfy 
12.11 or 12.21 it may be necessary to decompose the space fi" into “states” so that the conditional distributions 
have these properties. In addition, because different coordinates may have different marginal distributions, for 
12.11 to hold it may be necessary to partition the set [n] of coordinates. 


2.2. Homogeneity. The main result of this section shows that by partitioning the space Q” and/or the set \n\ of 
coordinates it is always possible to “approximate” a given measure p by measures that satisfy 12.11 for some suitable 
p as well as 12.21 . In fact, the number of parts that we have to partition [n] and Q" into is bounded only in terms of 
the desired accuracy but independently of n. 

Let us introduce some terminology. If V - (Vi,..., 14) is a partition of some set V, then we call #V = k the size 
of V. Furthermore, a partition W — [W\,... , W/) refines another partition V = (14,..., Vjt) if for each i e [Z] there is 
j e [fc] such that Wi c Vj. 
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For e > 0 we say that /i e 0P{Q. n ) is e-regular on a set [7 c [n] if for every subset Sc U of size |S[ > e\U\ we have 

<||<r[ - |S] - tr[ -1(7 ]IItv)/j < £ - 

Further, p is e-regular with respect to a partition F if there is a set / c [#V\ such that T.ia[#v]\j\^i\ < ew and such 
that /i is £-regular on V,- for all /' e /. Additionally, if F is a partition of [ n\ and S is a partition of Q", then we say that 
p is £ -homogeneous with respect to (F, S) if there is a subset 7 c [#S] such that the following is true. 

HM1: We have /i(S;) > 0 for all i e 7 and Z;e[#s]\jp(S<) < £. 

HM2: for all i e [#S] and j e [#F] we have max^o-^. || <7[ • | Vj] - a'[ • | Vj ] || w < e. 

HM3: for all i e 7 the measure p[ • |S,] is £-regular with respect to F. 

HM4: p is £-regular with respect to F. 

Theorem 2.1. For any £ > 0 there exists N - N{e,Q.) >0 such that for every n > N, any measure /i e ,'¥(£1") and any 
partition Fo of \ n \ of size # V o < l/£ the following is true. There exist a refinement V ofV o and a partition S ofD.' 1 
such that#V + #S< N andsuch that p is e-homogeneous with respect to (V, S). 

Informally speaking, Theorem l2.1l shows that any probability measure p e 3^{Ft n ) admits a partition (F, S) such 
that the following is true. Almost the entire probability mass of p belongs to parts S; such that the conditional 
measure p[- |S ; ] is £-regular w.r.t. F. This means that almost every coordinate x e [n] belongs to a class Vj such 
that for every “large” U c Vj for a chosen from p\■ [S;| very likely the empirical distribution a[-\U] is close to the 
marginal distribution (<r [ • [ ^/] )^[. |S ] °f the entire class. 

Theorem l2.1l and its proof, which we defer to Section [231 are inspired by Szemeredi’s regularity lemma l40l . Let 
us proceed to state a few consequences of Theorem l2.ll 
A (£, k ) -state of p is a set Sen” such that p(S) > 0 and 

— E ||/tj{Xl.JCjfc} [ ■ IS] - PJjci [• |S] ® • •• ® P[x k [• |S] Ijy < £. 

n xi,...,Xj fc£[n] 

In other words, if we choose xi,..., £ [n\ independently and uniformly at random, then the expected total vari¬ 

ation distance between the joint distribution pp Xl ,. |S] of xi,...,Xjt and the product p^ Xl [■ |S] »•••«> p^ x . [■ |S] 
of the marginal distributions is small. 

Corollary 2.2. For any e > 0, k>2 there exists tj = 77 (£, k, Q) > 0 such that for every n > 1 Ip any measure pe g? (Q”) 
has pairwise disjoint (£, k)-states S\,...,Sn such that p{Si) > 77 for all i e [IV] andj^^ =l p(Si) > 1 - £. 

Thus, we can chop the space Q” into subsets Si,..., Sn, N < 1 Iq, that capture almost the entire probability mass 
such that p[- |Sj] “resembles a product measure” for each i e [N]. We prove Corollarv l2.2l in Section [2~ll 
Let us call p (£, k) -symmetric if S = Q” itself is an (£, k ) -state. 

Corollary 2.3. For any £, k there exist S, r/> 0 such that for all n > Hr) and all pe .5^(0”) the following is true. If for 
any two ( S, k) -states Si, S 2 with p(Si), PIS 2 ) >r) we have 

- E llm[-|Si]-p u [-|S 2 ]|| xv <d, (2.3) 

11 xe[n] 

then p is (£, k ) -symmetric. 

Thus, the entire measure p "resembles a product measure” if extensive states have similar marginal distributions. 
Conversely, we have the following. 

Corollary 2.4. For any £ > 0 there isy> 0 such that for any 77 > 0 there exists 5 > 0 such that for all n> 1/8 and all 
p e 2?{Fl n ) the following is true. Ifp is ( 8 ,2) -symmetric, then for any (y, 2) -state S with p{S ) >77 we have 

— E IImuI"is]— p\x ||tv < e - 

n XE [n\ 

The proofs of Corollaries l2.3l and l2.4l can be found in Sections l2.5l and [2j6l respectively. Finally, in Section [2?7l we 
prove the following fact that will be useful in Section[4] 

Proposition 2.5. For any £ > 0 there exist 8 > 0 such that for large enough n the following is true. If p e pjfi/kl' 1 ) is 
(8, 2 ) -symmetric, then p® pe £?{Q. n x Q n ) is (£, 2 ) -symmetric. 
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2.3. Proof of Theorem l2.1 1 Throughout this section we assume that n is sufficiently large. To prove Theorem l2.ll 
and guided by (40) , we define the index of p with respect to a partition F of [n] as 

ind M (F) = -J—E E E ((<T[co\x]-<Tla)\Vj}) 2 ) 

l i2 l n wen j£[#V] xzVj 

The index can be viewed as a conditional variance (cf. ED). Indeed, choose x £ [n] uniformly and independently 
of < 7 . Furthermore, let be the tr-algebra generated by the events {x e V] \ for i e \#V\. Writing E[• ] and Var[-] for 
the expectation and variance with respect to the choice of x only, we see that 

ind^F) = -i- E E(Var[a[w|x]|^v]> M . 

I i2 l wen 

Lemma2.6. For any partition V of[n] we have\nd^{V) e 10,11. IfW is a refinement of V , then ind^(lF) < ind^(F). 

Proof. The fact that ind^fF) e [0,1] is immediate from the definition. Moreover, if W refines V, then PFy c PF W . 
Consequently, E(Var[<7Mx]|#w])/i £ E<Var[rr[w[x|177V11^. Averaging over coe D yields ind /; (lV) < ind^(F). □ 

Lemma 2.7. If pc fails to be e-regular with respect to V, then there is a refinement W ofV such that#W < 

2#F andind^W) < ind^(F) - e 4 /(4|Q| 3 ). 

Proof. Let ] be the set of all indices j e [#F] such that there exists S c Vj of size |S| > e\Vj\ such that 

(| o' [ • IS] — <x [ -1 Vy ] ItvE 5 e. (2.4) 

Since p fails to be e-regular with respect to F we have 

E l^/l - En - (2.5) 

For each j £ J pick a set Sj c Vj, \Sj \ > e| Vj | such that 12.41 is satisfied. Then there exists 0 )j e Q such that 

<|<r[ft»;|S ; ] > £/(2|n|). (2.6) 

Let W be the partition obtained from F by splitting each class Vj, j e /, into the sub-classes Sj, Vj \ Sj. Clearly, 
#W < 2#V. Furthermore, 

= E E(Var[<7Mx]|JV]E = ^ E (E(Var[ < 7[w|x]|^]> M + E(Var[E[<7[w|x]|^]|^]> fi ) 

l i2 l (oeQ (oen 

= ind M (W) + -l- E F(Vama[w\x]\& w ]\& v ])n- (2-7) 

l i2 l wen 

If j E / then 12.61 implies that on Vj we have 

I S ’ I £ 3 

{Vn[E[o[wj\x)\& w )\&v])^>^{la[i0j\Sj]-0[Wj\Vj]) 2 )^>^p. ( 2 . 8 ) 

Hence, combining 12.51 and 12.81 , we find 

1 e 4 

— E E<Var[E[<7[M*]|i%]|^v]E> —- 3 . (2.9) 

l i2 l wen 4 l i2 l 

Finally, the assertion follows from 1 12.71 and 12.9L □ 

Proof of Theorem [2J\ The set is compact. Therefore, there exists a partition Q = (Qi,..., Qk) of 2?{Fi) into 
pairwise disjoint sets such that for all i £ [ 1 C] and any two measures p, p' £ Qi we have || p - p' || |v < £. 

Given any partition W of \n\, we can construct a corresponding decomposition S(IF) of Q" as follows. Call 
a,a' £ fl n W-equivalent if for every i £ \#W\ there exists j £ [#Q] such that a[-\Wi\,cr'[-\Wj] £ Qj . Then S(LF) 
comprises of the equivalence classes. 

We construct the desired partition V of [n] inductively, starting from any given partition V (0) of size at most 
l/£. The construction stops once p is £-homogeneous with respect to (F( t), S (F( ?))). Assuming that this is not the 
case, we obtain F(f + 1) from F( t) as follows. If p fails to be £-regular with respect to V(t), then we let F(r+ 1) be 
the partition promised by Lemma l2?7l which guarantees that 

#F(r+l)<2#F(r) and ind M (F(r+ 1)) < ind^(F(fl) - £ 4 /(4|Q| 3 ). (2.10) 
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Otherwise let S(f) = S(F(f)) and s(f) = #S(f) for the sake of brevity. Further, let p,- jt = yu[ • |Sj(f)] for i e [s(f)] with 
p[S/(f)] > 0. Moreover, let /(f) be the set of all i e [s( t)'\ such that /i[S;(f)] > 0 and yu/, f fails to be £-regular with 
respect to V{t). If/i fails to be £-homogeneous with respect to (F (f), S (f)) but/i is £-regular w.r.t. V(t), then 

E /i[S,(f)]>e. (2.11) 

iel{t) 

Lemma l277l shows that for any i e /(f) there exists a refinement W(f, /) of V(f) such that 

ind Ai . t (Mf(f,i))<ind Ai . ( (F(f))-£ 4 /(4|0| 3 ). (2.12) 

Let V(f + 1) be the coarsest common refinement of all the partitions (W(f, i)),- e /(f). Then 

#F(f+l)<#F(f)-2 #<? . (2.13) 

In addition, 12.121 and Lemma [2T6l imnlv 

ind Wit (V(f + D) < ind w ,(F(f)) - Hi £ /(f)}£ 4 /(4|D| 3 ). (2.14) 

Therefore, by 12.111 , 12.141 and Bayes’ rule 

indfi(V(f +1)) = — Y, E E ({<T[(o\x]-a[(o\Vj(t+mf) u 

men j£[#V(t+l)]x£Vj(t+l) M 

= -^T7 E E p[S i -(t)](( 0 '[w|x]-<T[w|Vj(f+l)]) 2 ) 

<oj,x£e[.s(f)]:/4S;(f)]>0 I,f 

= E /i[S f (f)]ind Rt (V(f+l)) 

<-£ 5 /(4|0| 3 )+ £ Ai[S;(f)]ind M ,. t (F(f)) = ind |U (lf(f))-£ 5 /(4|Q| 3 ). (2.15) 

i:jU[Si(£)]>0 

Combining 12.101 . 12.151 and I.emma iZhl we conclude that p is £-homogeneous with respect to (V(T),S{T)) for 
some T < 4|Q| 3 /£ 5 . Finally, 12.131 entails that #V{T),#S(T) are bounded in terms of £,Q only. □ 

2.4. Proof of Corollarv l2.21 To derive Corollary [2(2] from Theorem l2.1l we use the following handy sufficient con¬ 
dition for (£, k) -symmetry. 


Lemma 2.8. For any k > 2, £ > 0 there isS — S (£, k, L2) such that for large enough n the following is true. Assume that 
p E SA(Q. n ) isS-regular with respect to a partition V andsetp.il-) = {a[- 1V/])^ for i e [#V], If 

E — (Wtri-WP-PiW^) <6, (2.16) 

i€[#V] n 


then p is (£, k ) -symmetric. 


Proof. Choose a small f = <f(£, fc,L2) > 0 and a smaller 5 — <?(£) > 0. Then 12.161 implies that there is / c \#V] 
satisfying 

ElV/|>(l- 0 « (2.17) 

/£/ 

such that for all j e J, S c Fp |S[ > Vj \ we have 

<||<r[-|S]-p J || T vE<^ (2.18) 

In particular, we claim that 12.181 implies the following (if £ is small enough): 

Vwcfi.j e/,ZcQ" : p(I) > < 1/4 => |{xe F y : |<a[w|x]|2>^-pjM| > ^ 1/4 }| < £ 1/4 |Fj|. (2.19) 

Indeed, assume that (1 {<teX})^ > £ 1/4 and |{xe Vj : |(<T[wo|Jt]|2)yi-P;(t<Jo)| >£ 1/4 }| > f 1 / 4 |f / /l for some a >o £ Q. 
Then because (< 7 [-|x]|Z)p is a probability measure on Q for every x, there exists to e Q such that the set S = 
{xe Vj : <cr[£<j|x] |2)>^ < pjlco) - <f 1 / 4 /|Q|} has size |S| > £ 1/4 | V)|/(2|Q|). In particular, <o-[w|S]|2)^ < pjpo) - £ 1 / 4 /|L2|. 
Therefore, by Markov’s inequality 


(l{(T[to\S]>Pjlco)-^ l 3 }\Z)^< 


Pjl0))-e l4 l IQI 

p/(«)-f 1/3 


< 


w 1/4 /ioi 

w i/3 


< l-£ 1/4 /(2|Q|). 






















Consequently, we obtain 


(||o-[- IS] - fij > f 1/3+1/4 <1 {a e Z}> /1 /(2|Q|) > f 718 . 

Since |S| > £ 1/4 [ Vy|/(2|Q|) > f\Vj\, this is a contradiction to 12.181 . 

Now, fix any e O and let xi,...,xjfc £ [n] be chosen independently and uniformly at random. Let Z/ 2 = 

Z/j C.v|,..., Xh) c L2” be the event that cr(x 2 ) = aq for all i < h. We are going to show that for 0 < h < k, 

E[p(Z ft )|<o-[w ft+ i|x ft+ i]|Z ft > M -<o-[w h+ i|x/ !+ i]) jU |] <£ 1/5 . (2.20) 


In the case h — 0 there is nothing to show. As for the inductive step, condition on xi,...,x/j. 

Case 1: p(Z/ 2 ) < f 1/4 : regardless of the choice of x/ 1+ | we have 

p(Z/j)|<<T[w,j + i|x; 1+ i]|Z; i > /i -<0-[W/ 2+ i|x/ 2+ i) |t( | <f 114 . 

Case 2: p(Z/ 2 ) ><f 1/4 : due to (ZTT) with probability at least l-2f we have x/ 1+ i e Vy \ {xi.x/,} for some j £ /. 

Hence, 12.191 implies E Xh+1 [|(< 7 [w; 2 + i|x/ 2 + i]|I / 2 > /2 - <<r[w; 2+ i|x; 2+ i]> fl |] <£ 1/4 . 

Hence, 12.201 follows. 

To complete the proof, we are going to show by induction on h £ [k\ that 


h \ h 

Y[a[o)i\Xi]) -nUrMxt]^ 


i= 1 


i= 1 


< he 15 . 


For h = 1 there is nothing to show. To proceed from h to h + 1 we use the triangle inequality to write 


h+l \ h+1 

II a[(t)i\xi\ ) - n (ctwilxiDju 


i=l 


(=1 


<E [jU(X; 2 ) | (<T [W , 2+ 1 |x, 2+ i] |Z^>^ - (<7[W; 2+ 1 Ix/j+il)^ | ] 


+ E 


(tr[<u^ + i|x/ 2+ i])^ 

Invoking the induction hypothesis and (2.20t completes the proof. 


i=i 


f[ a[o)i\Xi\ ) - n (o-[(Oi\Xi])i 


i=i 


( 2 . 21 ) 


□ 


Proof of Corollarv \2.2\ For a small enough S - 8(e, k) > 0 let ( V, S) be a pair of partitions of size at most N = N{8, Q) 
such that p is 5/2-homogeneous with respect to (V, S) as guaranteed by Theorem 12.II Let T] — eI(2N) and let / be 
the set of all j £ [#S] such that p(Sy) > 7] and such that p[ ■ |Sy] is 5-regular with respect to V. Then 

y p(Sy) < 5 +e/2 < e. 
je[#S]\/ 

Furthermore, for every j £ J the measure /i[- |Sy] satisfies 12.161 due to HM2. Therefore, Lemma [2781 implies that 
ju[-|Sy] is (e, fc)-symmetric. Consequently, the sets (Sy)y e y are pairwise disjoint (e, k)-states with /i(Sy) > T) for all 
j £j and £y e y jtf(Sy) > 1 - £. □ 

2.5. Proof of Corollarv l2.31 Pick small enough 8 — <?(£, fc,£T),y = y(5),r/(y) > 0. Then by Theorem 12. 1 1 u is y- 
homogeneous with respect to [V, S ) for partitions that satisfy #V + #S < N - IV(y). Let / c [#S] contain all j such 
that/i[-|Sy] is y-regular with respect to V and such that/i(Sy) > 77 . Let yi;y = (u| • IL/ l>^|.|,sy ■ Then by HM2 for every 
j e J we have 

l E , s <3y. 

“ ie[#V] 1 

Therefore, l.emma [2~8l imnlies that Sy is a (8, 2)-state. Consequently, our assumption 12.31 and the triangle inequal¬ 
ity entail that for all j, j’ £ J, 
























Choosing 77 small, we can ensure that Y.jgj p(S/) < 5. Therefore, letting p; = (a\-\ V)])^, we obtain from 12.221 


E ^<lkMV/]-p/|| w > <5 + E ^EWdlal-IVil-A/lwEr.is,, 


ie[#V] n 


ie [#VI n jej 
Wi\ 


<28+ £ '-fZtilS]) 

ie[#V] “ ye/ 

<55. 


[by HM2] 


(2.23) 


Since p is y-regular and thus 55-regular w.r.t. V by HM4, 12.231 and Lem ma [2~isl imply that p is (£, k) -symmetric. 


2.6. Proof of Corollarv l2.41 Choose a small y = y(£, Q) and a smaller 5 = 5(y,r;). Assume that S is a (y, 2)-state with 
p(S) > 77 and that p is (5,2) symmetric. Assume for contradiction that 

- E IImIxI* 15] — ||tv > (2-24) 

n xeln] 


Let 


W={xeV:l f i lx l-\S\-tii x l-]]l rv >el2} and 

W s (w) = {xzWi :s-(jttj x [ft)|S] -pj*M) > e/(2|0|)} for w e Q, se {±1}. 

Then 12.241 entails that |W| > ent 2. Therefore, there isweQ such that W s (a/) | > en/(2|Q|) for either 5 = +1 or 
5 = -1. Let W’ — W s U») for the sake of brevity. Of course, by the definition of W ’, 

(<<rM W']>^, S] - <<7M W]) J 2 > ^ (2.25) 

Moreover, because S is an (y,2)-state, the measure p[ • |S] is (y,2)-symmetric. Therefore, 


2 \ 

(tr[a>|W']-(T[w|W']) fJ[ . |S] ) \ =tt —2 E [(o-[w|x]a[w|y]) M . |S] -<T[w|x]> (U[ .|s](T[w|y]) jU| . 

< U \-1 SI I*" \ yvpW 1 

[as | W'| > £«/(2|Q|)]. 


//[•IS] i" 1 x.yaW' 
4y|Q | 2 


IS] 


Similarly, since p is (5,2)-symmetric, 

2 \ ^ 

(a[a)\W']-(T[a>\W']) ) ) = ——2 £ [(a-Mx]<7My]) — <t[oj|jc]) m < r[a>[y]) 

' // 11/1/1 jc.yeW' 

On the other hand we have 


45|0 | 2 


(a[a/| W'] - ( T [<u| W']> J 2 \ > p(S) / (<7[w|fV'] - (tM W , ])J 


//[•IS] 


>P(S) 


i (<t[ W | W']) M| . |S] - <tM W']> m ) 2 - / [a[o>\W] - 


//[•I S]j 


Finally, plugging 1 12.251 , 12.271 and 12.261 into 12.281 , we find 


45|Q | 2 


>77 


e 2 4y|0 | 2 


8 | Q | 2 £ 2 


which is a contradiction if 5 is chosen small enough. 


(2.26) 


(2.27) 


(2.28) 


2.7. Proof of Proposition l2.51 Choose small enough a - a{c, Q), y = y(a) > 0, y = y(y) > 0 and an even smaller 
5 = 5(y,y) > 0 and assume that p is (5,2)-symmetric. Suppose that p is y-homogeneous with respect to a partition 
(17, S) such that #17 + #S < N — N{j) as promised by Theorem l2.ll Let / be the set of all j e [#S] such that p(Sy) > 
j 2 IN. Moreover, let I be the set of all i e [#V] such that p is y-regular on Vi and 17,1 > jn/N. By Coroll arv !2.4l we 
have 

7777 E WPixl-lSj]-m x [-]\\ rv <r forallie/,/£/, 

1 jceVi 
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(2.29) 


provided that 8 is chosen small enough. Therefore, letting p; = (a\-\ Vp) )n for all i e I we have 

dlal-l^]-Pi|I tv) m <27- 

Fix some i e I. We claim that p® p is a-regular on V,. ffence, let U c l// be a set of size \ U\ > a\ Vp and let 

«f = {|| < 7[-|C7]-pi|| TV < r 1/3 }. 

Then 12.291 implies that (1 {a £ *%'})^ < y 1/3 , because p is y-regular on V-,. Now, fix some a e S. For at e Q let 
U[a,co ) = {x e U : cr(x) = at}. Let 

S'(a, cu) = {|r[- |[/(ct,«)] - pi l-jy < y 1/3 }. 

If | U(a, at) | > y l/2 |(J[, then due to 12.291 and y-regularity we obtain, by a similar token as previously, (1 {t £ <S'{a,o>)\)^ < 
y 1/3 . Consequently, the event <§'{(T) that §'{o,at) occurs for all at satisfying \U(a,a>)\ > y l/2 |(i has probability at 
least 1 - |Q|y 1/3 . Therefore, for any at,at' eQ we obtain 


— Y Ifo'(x) = ftj}l{r(x) =oj'}-pi(w)pi(oj') 
1^1 xeU 


< (|n| + l)y 1/3 + 


— Y l{o'W = w}l{T(x) = w'}-pi(w)pi(w') 
xaU 


| a e S,t e £'{(t) 


, 1/8 


<y 1/7 + / max |t[w'|[/( 0-,W)] - Pi(tl/)| lo- E ^,T E^'to-) ) <y 
\ Cl):| t/(o",w) | >y 1/2 1 f/| 

Summing over all at, at' and choosing y small enough, we conclude that p ® p is a-regular on 
Finally, 12.291 implies that p ® p satisfies 

< || (<r® t) [-1V/] - pi ® p; | w )^ < a. 

Therefore, picking a small enough, we can apply Lemma [2dil to conclude that p ® p is (e, 2)-symmetric. 


3. Factor graphs 


3.1. Examples. The aim in this section is to set up a comprehensive framework for the study of “random factor 
graphs” and their corresponding Gibbs measures. To get started let us ponder a few concrete examples. 

In the Ising model on a graph G- ( V, H) the variables of the problem are just the vertices of the graph. The values 
available for each variable are ±1. Thus, an assignment is simply a map cr: V —► {+1}. Moreover, each edge of G 
gives rise to a constraint. Specifically, given a parameter ft > 0 we define a weight function y/ e corresponding to the 
edge e-{v,w} by letting i// ( ,(<r) = exp(/Lj(z/p7( w)). Thus, edges e - { v, w\ give larger weight to assignments cr such 
that a[v) = a{w) than in the case cr( v) ^ cr[w). The corresponding partition function reads 


Zp(G)= Y I\VeM= Y ex P 

a:V^{±l}eeE a:V^{± 1} 


p Y v(v)a(w) 

{v,w}eE 


Further, the Gibbs distribution /iG (i e induced by G, /3 is the probability measure on l±l} v defined by 

1 t— r 1 




exp 


p Y 

{v,w}eE 


Thus, nc,p weighs assignments according to the number of edges e-{v,w] such that o{v) — cr(ut). 

The Ising model has been studied extensively in the mathematical physics literature on various classes of graphs, 
including and particularly random graphs. For instance, if G(n, d) is a random regular graph of degree d on n ver¬ 
tices, then Zp (Gin, d)) is known to “converge” to the value predicted by the cavity method |TB]. Formally, the cavity 
method yields a certain number F(/3, d) such that 

lim -E[lnZfl(G(rc,d))] =F(jM). (3.1) 

Cl—oo n e 

Because Zp(G(n,d)) is exponential in n with high probability, the scaling applied in 13.11 is the appropriate one 
to obtain a finite limit. Furthermore, by Azuma’s inequality In Zp{G{n,d)) is concentrated about its expectation. 
Therefore, 13.11 implies that jjlnZp{G{n,d)) converges to Fif’t,d) in probability. 
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The Potts antiferromagnet on a graph G - ( V, E) can be viewed as a twist on the Ising model. In this case we look 
at assignments cr: V —► [fc] for some number k > 3. The weight functions associated with the edges are defined by 
i jj e {cr) = exp (-/ilicit id = for some p > 0. Thus, this time the edges prefer that the incident vertices receive 

differentvalues. The Gibbs measure and the partition function read 


= z„(G) “ P 

-f Y l{cr(^) = cr(ir/)} 

. Zp(G) = Y ex P 

~P Y 1 {(7[v)-a[w)} 

{v,w}eE 

cr.V—> [k] 

{v,w}eE 


While it is known that lim^oo -^Ejln Z^(G(rc, d))] exists and that In Zp{G[n,d)) is concentrated about its expec¬ 
tation m the precise value remains elusive for a wide range of d, (’> (in contrast ferromagnetic version of the 
model (20)). However, it is not difficult to see that for sufficiently large values of d, (’> we have (To) 

lim -E[lnZfl(G(n,d))] < lim -InE [Za(G{n,d))\. 

72—00 yi H 72—00 Yl H 

Hence, just like in the random /c-SAT model the first moment overshoots the actual value of the partition function 
by an exponential factor. The Potts model is closely related to the fc-colorability problem. Indeed, if we think of the 
k possible values as colors, then for large p the Gibbs measure concentrates on colorings with few monochromatic 
edges. 

As a third example let us consider the following version of the random /c-SAF model. Let k > 3, A > 1 be fixed 
integers, let V n = {x\,...,x n } be a set of Boolean variables and let d n : V n x {+1} —► [A] be a map such that 

m= Y (d n {x, 1) + d n lx, -l))/fc 

x<iV n 

is an integer. Then we let <t>(n,k,d n ) be a random fc-CNF formula with m clauses in which each variable x e V n 
appears precisely d n {x, 1) times as a positive literal and precisely d n {x, — 1) times as a negative literal. As in Sec- 
tionQ] for a clause a and a truth assignment o : V —► {0,1} we let i// fl ((i) = exp(-/3l{cr violates a}). Then for a given 
parameter p > Owe obtain a Gibbs measure that weighs assignments by the number of clauses that they violate 
and a corresponding partition function Zp{<t>{n,k,d n )), cf. 11.11 - 11.21 . Hence, for given /) > 0, k > 3 and degree 
assignments [d n ) n the problem of determining lini^oo ^E[ln Zp[<S>[n, k, d n ))\ arises. This question is anything but 
straightforward even in the special case that d n {x,± 1) = do is the same for all x. In 0 we show how the results of 
the present paper can be put to work to tackle this case. 


3.2. Random factor graphs. The following definition encompasses a variety of concrete models. 


Definition 3.1. Let A > 0 be an integer, let LI, & be finite sets and let 'P = {i// 1 ,..., i/// }■ be a finite set of functions 
ipi :Ll hi — (0,oo) ofarityhi e [A]. A (A,L2, ¥,0) -model Jl = ( V,F,d , t,(y/ a ) a€ p) consists of 
Ml: a countable set V of variable nodes, 

M2: a countable setF of constraint nodes, 

M3: a map d:VuF ^ [A] such that 

Y d(x) = Y d ^’ (3-2) 

xeV azF 

M4: a map t : Cy u Cf — 1 • 0, where we let 

Cy - U W x Cf = U x 

ieV aeF 


such that 

11 _1 (0) n Cy| = 11 _1 (0) n Cf| for each 9 eO, (3.3) 

M5: a mapF —^ x ¥,a^yr a such that y/ a : Ll dM — (0,oo ) for all ae F. 

The size of the model is#JL -\V\. Furthermore, a .M -factor graph is a bijection G: Cy —> Cf, (x, i) G(x, i) such 
that t{G{x, i)) = t[x, i) for all (x, i) e Cy- 


Of course, 13.21 and 13.31 require that either both quantities are infinite or both are finite. 

The semantics is that A is the maximum degree of a factor graph. Moreover, fl is the set of possible values 
that the variables of the model range over, e.g., the set {+1} in the Ising model. Further, 0 is a set of “types”. For 
instance, in the random k-SAT model the types can be used to specify the signs of the literals. Additionally, 'V is a 
set of possible weight functions. 


li 







A model Jl comes with a set V of variable nodes and a set F of contraint nodes. The degrees of these nodes are 
prescribed by the map d. Just like in the “configuration model” of graphs with a given degree sequence we create 
d{v) “clones” of each node v. The sets Cy, Cf contain the clones of the variable and constraint nodes, respectively. 
Further, the map t assigns a type to each “clone” of either a constraint or variable node and each constraint node 
a comes with a weight function i j/ a . 

A ^"-factor graph is a type-preserving matching G of the variable and constraint clones. Let [,M) be the set 
of all ^-factor graphs and write G = G(M) for a uniformly random sample from Contracting the clones of 

each node, we obtain a bipartite (multi-) graph with variable nodes V and constraint nodes F. We often identify G 
with this multi-graph. For instance, if we speak of the distance of two vertices in G we mean the length of a shortest 
path in this multi-graph. 

For a clone (x, i) e Cy we denote by d(G, x, i) - G(x, i) the clone that G matches (x, i) to. Similarly, for (a, j) e Cf 
we write d(G, a, j ) for the variable clone (x, i) such that d(G, x, i) = [a, j ). Moreover, for a variable x we let d(G, x) = 
{d(G,x, i): i e [d{x]]} and analogously for a e F we set d(G, a ) = {d(G, a,j ): j e [d(a )]}. To economise notation we 
sometimes identify a clone (x, i ) with the underlying variable x. For instance, if <7 : V —* O is an assignment, then we 
take the liberty of writing cr(x, i ) = cr(x). Additionally, where convenient we view d(G, x) as the set of all constraint 
nodes a e F such that there exist i e \d(x)], j e [d{a]] such that (a,j) = G(x, /). The corresponding convention 
applies to d(G, a). 

A M -assignment is a map tr : V —>• Q and we define 

VG.aW = y/a[a(d G (a,l)),...,(T(dG(.a,d(a)))) foraeF, and t// G (cr) = ]“[ i// fl (cr). 

aeF 

Further, the Gibbs distribution and the partition function of G are 

Pg(o)=V'gWIZg, where Z(G) = £ t//g( o’). (3.4) 

a:V—*Cl 

We denote expectations with respect to the Gibbs measure by (-)g = <-)^ G . 

The fundamental problem that arises is the study of the random variable In Z (G). As mentioned in Section[l] this 
random variable holds the key to getting a handle the Gibbs measure and thus the combinatorics of the problem. 
The following proposition establishes concentration about the expectation. For two factor graphs G, G' e c fi (.M ) let 

dist(G, G') = |{(x, i) e Cy ■ d(G,x, i ) ^ d(G',x, ;)}|. (3.5) 

Proposition 3.2. For any A, Q, 0, T 1 there exists t)-tj{ A, Q, 0, TO > 0 such that for any (A, Q, T 7 ,0 )-model Ji of size 
n - #M > 1/77 and anye > 0 we haveP [|lnZ(G) - E[lnZ(G)]| > e] < exp {-rje 2 n). 

Proof There exists a number p > 0 that depends on A, fi, T 7 ,0 only such that for any two factor graphs G, G' e Vj (.M) 
we have | lnZ(G) - lnZ(G')| < p ■ dist(G, G'). Therefore, the assertion follows from Azuma’s inequality. □ 

Thus, Proposition [3T2] reduces our task to calculating the expectation E[lnZ(G)]. Generally, the standard first 
and second moment method do not suffice to tackle this problem because the logarithm sits inside the expectation. 
While, of course, Jensen’s inequality guarantees that 

E[lnZ(G)]<lnE[Z(G)], (3.6) 

equality does not typically hold. In fact, we saw examples where lnE[Z(G)] - E[lnZ(G)] is linear in the size H.M 
of the model already. If so, then the Paley-Zygmund inequality entails that ln(E[Z(G) 2 ]/E[Z(G)] 2 ) is linear in #Jl 
as well, dooming the second moment method. Furthermore, even if E[lnZ(G)] ~ lnE[Z(G)] the second moment 
method does not generally succeed 116]. Let us now revisit the examples from Section l3Tl 

Example 3.3 (the Ising model on the random d-regular graph). Suppose that d > 2, /) > 0. Let A - d, Q. - {±1}, 
T 7 = {y/}, where y/ : {+1 } 2 —* (0,oo), ( 04 , 02 ) >-► exp(/ 3 cric 72 ), and set 0 = {0}. Further, given n > 1 such that dn is 
even we define a (A,Q,T 7 ,©)-model M{d, n) by letting V - {xi,...,x n }, F = {a\,..., a^nn}, d{x) - d for all xeF, 
d{a)-2 for all a e F, t{x, i ) = t[f,j ) = 0 for all (x, i ) e Cy, (/,_/) e Cf, and y/ a -y for all ae F. Thus, all clones have 
the same "type” and all constraint nodes have arity two and the same weight function. Hence, the random graph 
Ci{.M) is obtained by matching the dn variable clones randomly to the dn constraint clones. If we simply replace 
the constraint nodes, which have degree two, by edges joining the two adjacent variable nodes, then the resulting 
random multigraph is contiguous to the uniformly random d-regular graph on n vertices. In the model Jl 13.61 
holds with (asymptotic) equality for all d, j3 T 8 j. 
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Example 3.4 (the Potts antiferromagnet on the random d-regular graph). The construction is similar to the previ¬ 
ous example, except that Q = [fc] is the set of colors and y/(cr \ , 02 ) = exp(-/3l{<j] = < 72 }). In this example 13.61 holds 
with asymptotic equality if either d < do (fc) or d > do( fc) and p < Paid, fc) for certain critical values do(k), Paid, k ). 
However, for sufficiently large d, p there occurs a linear gap ! [0 IT). 

Example 3.5 (random fc-SAT). To capture the random fc-SAT model we let A > 0 be a maximum degree and Q = 
0 = {±1}. Further, each s e {+l} fc gives rise to a function 

y/ s '■ l±l} fc —*■ (0,oo), a 1 —* exp (—/filer — — s}) 

and we let= {yr s : s e {+1 } fc }. The idea is that s is the “sign pattern” of a fc-clause, with s* = +1 indicating that 
the / th literal is positive/negative. Then a truth assignment <7 of the k variables is satisfying unless a ; = — s/ for all 
i. The corresponding model M has a set V = {xi,...,x n j of Boolean variables and a set F = {a\,...,a m } of clauses. 
Moreover, the map d : V —> [ A] prescribes the degree of each variable, while of course each clause has degree k. 
Additionally, the map t: Cy u Cf — 1 - 0 = {±1} prescribes the positive/negative occurrences of the variables and the 
sign patterns of the clauses. Thus, a variable x occurs |{i e [d(v )\: t(x, i ) = +1}| times positively/negatively and the 
7 th literal of a clause a is positive iff t{a,j ) = 1. Finally, the weight function of clause a is VHi(a,\ ),...,i{a,k))- The bound 
13.61 does not generally hold with equality (5JEI • 

While Definition l3. 1 l encompasses many problems of interest, there are two restrictions. First, because all weight 
functions ij/eT take strictly positive values, Definition 13.11 does not allow for “hard” constraints. For instance, 
Definition l3.1l does not accommodate the graph coloring problem, which imposes the strict requirement that no 
single edge be monochromatic. However, hard constraints can be approximated by soft ones, e.g., by choosing a 
very large value of p in the Potts antiferromagnet. Moreover, many of the arguments in the following sections do 
extend to hard constraints with a bit of care. However, the assumption that all y/ are strictly positive saves us many 
case distinctions as it ensures that Z(G) is strictly positive and that therefore the Gibbs measure is well-defined. 

The second restriction is that we prescribe a fixed maximum degree A. Thus, if we consider a sequence - 
{JlyPn of (A, Q, T,©)-models with #M n = n, then all factor graphs have a bounded degree. By comparison, if we 
choose a fc-SAT formula with n variables and m - anile clauses uniformly at random for fixed fc > 3, a > 0, then 
the maximum variable degree will be of order In nl lnln n. Yet this case can be approximated well by a sequence 
of models with a large enough maximum degree A. In fact, if we calculate E[lnZ] for any fixed A, then the A —<• 00 
limit is easily seen to yield the answer in the case of uniformly random formulas. Nevertheless, the bounded 
degree assumption is technically convenient because it facilitates the use of local weak convergence, as we will 
discuss next. 

Remark 3.6. For the sake of simplicity in 13.41) we definied the partition function as the sum over all 0 : V —<• Q. 
However, the results stated in the following carry over to the cases where Z is defined as the sum over all configurations 
of a subset of0 f c n v , e.g., all 0 that have Hamming distance at most an from some reference assignment <jq 

for a fixed a > 0. Of course, in this case the Gibbs measure is defined such that its support is equal to Y?//. 

3.3. Local weak convergence. Suppose that we fix A. Q.'P.0 as in Definition l3. H and that M — (.M n )n is a sequence 
of (A, n.^P,©)-models such that ,M n = {V n , F n , d n , t n , {il/ a )aoF n ) has size n. Let us write G- G{M n ) for the sake of 
brevity. According to the cavity method, lim^oo TE[lnZ(G)] is determined by the “limiting local structure” of 
the random factor graph G. To formalise this concept, we adapt the concept of local weak convergence of graph 
sequences [30] Part 4] to our current setup, thereby generalising the approach taken in [T9]. 

Definition 3.7. A (A, Q, 'F, 0) -template consists of a (A,Q, '¥,©)-model .M, a connected factor graph H e ( fi{.M) 
and a root ru, which is a variable or factor node. Its size is k.M. Moreover, two templates H, H' with models .M - 
{V, F, d, t, (if/ a )), HI' = (V', F' ,d', t ', ( 1 i/' a )) are isomorphic if there exists a bijection n: V u F —^• V' u F' such that 
ISM1: n(r H ) = r' H , 

ISM2: n(V) = V' andn(F) = F', 

ISM3: d(v) — d' (n(v)) for all v eV u F, 

ISM4: t(v, i) = t'(n(v), i) for all (v, i) e Cy u Cf, 

ISM5: if/ a = if/jz(a) for all ae F, and 

ISM6: if(v,i ) e Cy,(a,j ) e Cf satisfyd(G,x,i) - ( a,j ), thend(G’,n(x),i) - (n(a),j). 

Thus, a template is, basically, a finite or countably infinite connected factor graph with a distinguished root. More¬ 
over, an isomorphism preserves the root as well as degrees, types, weight functions and adjacencies. 
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Let us write [H] for the isomorphism class of a template and let © = 0 (A, fi, 0, *?) be the set of all isomorphism 
classes of (A, Q, 1 ?,©)-templates. For each [H] £ 0 and £ > I let d f \ H] be the isomorphism class of the template 
obtained by removing all vertices at a distance greater than £ from the root. We endow 0 with the coarsest topology 
that makes all the functions 


r e 0 - 1 {d e [f] = d { [r 0 ] } e {0,1} for £ > 1, F 0 £ 0 


continuous. Moreover, the space 2? (0) of probability measures on 0 carries the weak topology. So does the space 
5 s2 (0) of probability measures on ^(0). For F e 0 we write <5r £ &*{&) for the Dirac measure that puts mass one 
on the single point F. Similarly, for A £ £?(($) we let S\ £ & >2 {<S) be the Dirac measure on A. Our assumption that 
the maximum degree is bounded by a fixed number A ensures that 0, 2P (0), 2? 2 (0) are compact Polish spaces. 

For a factor graph G £ and a variable or constraint node v we write [G, v] for the isomorphism class of 

the connected component of v in G rooted at v. Then each factor graph G £ c £S(.M n ) gives rise to the empirical 
distribution 


Ag = 


1 

Wn\ + \F n \ 


E s lGjV] £&>m. 

veV„uF„ 


We say that Jl converges locally to f) £ £P(0) if 


lim E[5 Ac ] = S e . (3.7) 

72—00 

Denote a random isomorphism class chosen from the distribution d by T — Tg. Unravelling the definitions, we see 
that 13.71 holds iff for every integer £ > 0 and every [//1 e 0 we have 

--- V l{d^[G, v] =</[#]} n ^°° v\d e T d = d e {H]\ in probability. (3.8) 

\ V n\ + \Fn\ ve ^ uFn [ J 

We are going to be interested in the case that M_ converges locally to a distribution Q on acyclic templates. Thus, 
let T be the set of all acyclic templates. Further, we write V for the set of all templates whose root is a variable node 
and & for the set of all templates whose root is a constraint node. Additionally, for a template [ H\ we write r\u\ 
for the root vertex, d[H] for its degree and y/\n\ for the weight function of the root vertex if [H] £ 2P. Moreover, for 
j £ [d[H] ] we write | H\ f j for the template obtained from | I I \ by re-rooting the template at the j th neighbor of /'[//|. 
(This makes sense because condition ISM6 from Dehnition l3.7l preserves the order of the neighbors.) 

We will frequently condition on the depth-/ 7 neighborhood of the random factor graph G for some finite £. 
Hence, for G, G' £ and £ > 1 we write G =g G 1 if d ( \G,x\ - d f \G',x\ for all variable nodes x £ V n and 

d^ +1 [G, a] - y +1 [G',a\ for all constraint nodes a £ F n . Let 2T( = be the cr-algebra on t^(.^„) generated 

by the equivalence classes of the relation = f . Additionally, for G £ ( S(.M n ) and £ > 0 we let 


^G,e 


1 

|V„| + |F n | 


E ^d f [G,x] + E ^ d M [G,a] 

x€ V n a€F n 


be the empirical distribution of the depth-/ 7 neighborhood structure. 

Furthermore, let 

% e = [d e T: rETnF|u|d^ +1 r: reln^j. 

Then for a probability measure -d£g?(‘Z) we denote by f)/’ the image of f) under the map 

if TeTn7, 
if reln^. 

every £ > 1. Hence, 13.81 entails that converges 
for every £ > 1. (3.9) 


T-3>, 


d e T 

^+i' 


Because all degrees are bounded by A, the set is finite for 
locally to 0 £ ??{%) iff 


Urn E | A 6 i /-0/1 tv = ° 
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3.4. The planted distribution. While G is chosen uniformly at random (from the configuration model), we need 
to consider another distribution that weighs factor graphs by their partition function. Specifically, given £ > 0 let 
G( = G( i ji n be a random graph chosen according to the distribution 


P[G ( = G] = Z(G)-E 


1{G - G} 
E[Z| 3f\ 


(Ge<S{M n )), 


(3.10) 


which we call the planted distribution. The definition 13.101 ensures that the distribution of the “depth-/ 7 neigh¬ 
borhood structure” of G( coincides with that of G. 

Perhaps more intuitively, the planted distribution can be described by the following experiment. First step, 
choose a random factor graph G. Then, given G, choose the factor graph G/ randomly such that a graph G = ( G 
comes up with a probability that is proportional to Z(G). Perhaps despite appearances, the planted distribution 
is reasonably easy to work with in many cases. For instance, it has been employed successfully to study random 
fc-SAT as well as random graph or hypergraph coloring problems ITil9lfT1. 161f22 l. 


3.5. Short cycles. In most cases of interest the random factor graph is unlikely to contain many short cycles, and 
it will be convenient for us to exploit this fact. Hence, let us call a factor graph G l-acyclic if it does not contain a 
cycle of length at most l. We say that the sequence .M_ of models has high girth if for any £,l> 0 we have 

liminf P [G is Z-acyclic] > 0, liminf P [G( is Z-acyclic] > 0. (3.11) 

Thus, there is a non-vanishing probability that the random factor graph G is Z-acyclic. Moreover, short cycles do 
not have too heavy an impact on the partition function as the graph chosen from the planted distribution has a 
non-vanishing probability of being Z-acyclic as well. 

In the following, we are going to denote the event that a random factor graph is Z-acyclic by .s//. Let us highlight 
the following consequence of the high girth condition and the construction of the planted distribution. 


Proposition 3.8. Assume that Jl_ is a sequence of {A, Q,'F, 0) -models of high girth. Let (. > 1 he an integer and 
suppose that SB is an event such that P \G( e SB] = 1. Ifb is a real and l > 0 is an integer such that 


lim P[lnE[Z(G)|5>] > bn\s4i) = 1, (3.12) 


fZienlimn^ooElnElli^rW/lZtG)] > b. 

Proof. Since lim^ooP [G( e SB] = 1 the high girth condition 13.111 implies that lim^oo P [ Gp e SB\$#i\ — 1 for every 
Z. Set SBi-s^ir SB. Then by the definition 13.101 of the planted distribution, 


1 — o(l) — P [G^ e SB\s^j] — £ Z(G)E 

G&SSi 


MG= G} | 

= E 

1{GeS§/}Z(G)i 

- 

= E 

E[1{G e S&i}Z\2T(] | 

- IS&1 

E[Z|5>] 1 


E[Z|5>] 1 


E[Z|5>] 1 ‘J 


Consequently, P [E[1{G e ^}Z]|5}] > E[Z|5>]/2|af ; ] = l-o(l). Hence. GLOP yields 


P [lnE[l{G e SBi\Z\\ATf\ > bn - 1|^/] = 1 - o(l). 

Therefore, the assertion follows from 13.111 . □ 


Remark 3.9. Strictly speaking, the first condition in 13.111 ) is superfluous as it is implied by the second one. 


From here on out we assume that .Jl_ is a sequence of (A, O, 'P,0) -models of high girth that converges locally to 
{) e 1Z(‘X) and we fix A, Q, 0, 'F for the rest of the paper. 


4. The Bethe free energy 

In this section we present the main results of the paper. The thrust is that certain basic properties of the Gibbs 
measure entail an asymptotic formula for E [In Z(G)]. The results are guided by the physics predictions from f29l . 
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4.1. An educated guess. The formula for E[lnZ(G)] that the cavity method predicts, the so-called “replica sym¬ 
metric solution”, comes in terms of the distribution 9 to which M_ converges locally. Thus, the cavity method 
claims that in order to calculate E[lnZ(G)] it is not necessary to deal with the mind-boggling complexity of the 
random factor graph with its expansion properties, long cycles etc. Instead, it suffices to think about the random 
tree T — Tq, a dramatically simpler object. The following definition will help us formalise this notion. 

Definition 4.1. A marginal assignment is a measurable map p: T —>■ 1J ( \ | PA(Il J ), T i—>• p T such that 
MAI: p T e £?*(Q) for all TeV, 

MA2: pi D. dT ) andprij - Pr\j for all T e &,j e [dj], 

MA3: For allT we have 

H(p T ) + (\ny/ T {(r)) pT - max jlT(v) + (lni//j’(<r)) v : v e SA{Fl dr ) s.t. vjy = p T]j forallj e [d T ]}•. (4.1) 

Further, the Bethe free energy ofp with respect to 9 is 

p r T £ r 1 

= E [(1 - drMprW] + p - B \H{p T ) + (Ini//r(<r)) pr W \, (4.2) 

where, of course, E[ - ], P [ • ] refer to the choice of the random tree T -Tq. 

Thus, a marginal assignment provides a probability distribution pj on Q for each tree whose root is a variable 
node. Furthermore, for trees T rooted at a contraint node p j is a distribution on Q dr , which we think of as the joint 
distribution of the variables involved in the constraint. The distributions assigned to T rooted at a constraint node 
must satisfy a consistency condition: the /th marginal of pj has to coincide with the distribution assigned to the 
tree T } j rooted at the 7 th child of the root of T for every j e [dj-]; of course, T j j is a tree rooted at a variable node. 
In addition, MA3 requires that for T e & the distribution pr maximises the functional H (v) + (i//j’(o-)} v amongst 
all distribution v with the same marginal distributions as p-p. Furthermore, the Bethe free energy is a functional 
that maps each marginal assignment p to a real number. For a detailed derivation of this formula based on physics 
intuition we refer to Ell- 

Given a distribution 9 on trees, the cavity method provides a plausible recipe for constructing marginal assign¬ 
ments. Roughly speaking, the idea is to identify fixed points of an operator called Belief Propagation on the random 
infinite tree EH • However, this procedure is difficult to formalise mathematically because generally there are sev¬ 
eral Belief Propagation fixed points and model-dependent considerations are necessary to identify the “correct” 
one. To keep matters as simple as possible we are therefore going to assume that a marginal assignment is given. 

Remark 4.2. Because the entropy is concave, conditions MA2 and MA3 specify the distributions p-p for T e FA 
uniquely. In other words, a marginal assignment is actually determined completely by the distributions p j for T eV. 

For a marginal assignment p, an integer £ and a tree T e X^> n Y we define 

Pe,T = V[pT\d e T= T], 

Thus, p( is the conditional expectation of p given the first f layers of the tree. Finally, to avoid notational hazards 
we let pj, pcj be the uniform distribution on fl for all T e © \ ST. 

Lemma 4.3. For any £ > 0 there is£ 0 > 0 such that for all £ > £q we / 2 ai/eE[|| p ( g e T - pr|| |T e Y] < e. 

Proof. Define an equivalence relation =g on Tn Y by letting T =( T' iff d e T — d e T'. Then for any a> e Q the sequence 
of random variables X({T) = P( g c T [c 0 ) is a martingale with respect to the filtration generated by the equivalence 
classes of =(. By the martingale convergence theorem (23] Theorem 5.7], (pf)p converges fi-almost surely to p. □ 

4.2. Symmetry. In the terminology of SectionEl the cavity method claims that yj H[In Z(G) | converges to the Bethe 
free energy of a suitable marginal assignment iff 

lint P [pa is (£,2)-symmetric] = 1 for any £> 0 (see (29]). (4.3) 

This claim is, of course, based on bold non-rigorous deliberations. Nonetheless, we aim to prove a rigorous state¬ 
ment that comes reasonably close. 
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To this end, let p be a marginal assignment. We say that is p -symmetric if for every e > 0 there is £q > 0 such 

that for all £ > £q we have 


lim P 

n—*oo 


~2 E \\l^GHx,y} - P( : gf[G,x]^ P(,d f [G,y]\\ T ,r > E 
n x,yeV„" " 1V 


= 0. 


(4.4) 


In other words, for any e > 0 for £ sufficiently large random factor graph G enjoys the following property with 
high probability. If we pick two variable nodes x, y of G uniformly and independently, then the joint distribution 
PGi{x,y) is close to the product distribution p f x j 8 Pf d f \Gy\ determined by the depth-/’ neighborhoods of x,y. 
Of course, as G has bounded maximum degree the distance between randomly chosen x, y is going to be greater 
than, say, lnln n with high probability. Thus, similar in spirit to 14.31 . 14.41 provided that far-apart variables typically 
decorrelate and that p captures the Gibbs marginals. 

In analogy to 14.41 . we say that the planted distribution of.M is p -symmetric if for every e > 0 there is £q > 0 
such that for all £ > £q we have 


lim P 

n—*o o 


— E 

„2 

n x,yeV„ 


l^GAix.y) P(,d e lG(,x] ® P(,d e {G e ,y] 


> £ 


= 0 


for any e > 0. 


The main result of this paper is 


Theorem 4.4. IfM_ is p-symmetric, then 

lim sup — E[lnZ(G)] < SSfi(p). 

n—> oo VI 

If the planted distribution ofis p -symmetric as well, then 

lim —E[lnZ(G)] = SBnlp). 

n ~*oo n 

Thus, the basic symmetry assumption 14.41 implies that SBoip) is an upper bound on ^H|lnZ(G)|. If, addition¬ 
ally, the symmetry condition holds in the planted model, then this upper bound is tight. In particular, in this case 
^E[lnZ(G)] is completely determined by the limiting local structure d and p. 

The proof of Theorem 14.41 which can be found in Section 14.61 is based on Theorem 12.11 the decomposition 
theorem for probability measures on cubes. More precisely, we combine Theorem l2.1l with a conditional first and 
a second moment argument given the local structure of the factor graph, i.e., given PTf for a large £. The fact 
that it is necessary to condition on the local structure in order to cope with “lottery effects” has been noticed in 
prior work l6l fl5llT8l[T9l . Most prominently, such a conditioning was crucial in order to obtain the precise A:-SAT 
threshold for large enough k [22]. But here the key insight is that Theorem l2. 1 l enables us to carry out conditional 
moment calculations in a fairly elegant and generic way. 

The obvious question that arises from Theorem 14.41 is whether there is a simple way to show that Jf_ is p- 
symmetric (and that the same is true of the planted distribution). In Sect i o n s |4. 3 1 a n d [4~4l we provide two sufficient 
conditions called non-reconstruction and Gibbs uniqueness. That these two conditions entail symmetry was pre¬ 
dicted in [29], and Theorem l2. 1 l enables us to prove it. 


4.3. Non-reconstruction. Following [29] we define a correlation decay condition, the “non-reconstruction” con¬ 
dition, on factor graphs and show that it implies symmetry. The basic idea is to formalise the following. Given e > 0 
pick a large £ — £{e) > 1, choose a random factor graph G for some large n and pick a variable node x uniformly at 
random. Further, sample an assignment <r randomly from the Gibbs measure pc- Now, sample a second assign¬ 
ment r from pc subject to the condition that r(y) = tr(y) for all variable nodes y at distance at least £ from x. Then 
non-reconstruction condition asks whether the distribution of t(x) is markedly different from the unconditional 
marginal Pg\x- More precisely, non-reconstruction occurs if for any e there is £[e) such that with high probability 
G is such that the shift that a random “bounary condition” a induces does not exceed e in total variation distance. 

Of course, instead of conditioning on the values of all variables at distance at least £ from x, we might as well 
just condition on the variables at distance either £ or £ + 1 from x, depending on the parity of £. This is immediate 
from the definition 13.41 of the Gibbs measure. 

As for the formal definition, suppose that G e is a factor graph, let x e V n and let £ > 1. Fet SJ f (G,x) 

signify the cr-algebra on O" generated by the events 1 'rr(y) = w} for w e Q and y e V n at distance either £ or £ + 1 
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from x. Thus, Vy (G, x) pins down all <r(y) for y at distance £ from x if £ is even and £ + 1 otherwise. Then we say 
that Jl_ has non-reconstruction with respect to a marginal assignment p if for any e > 0 there is £ > 0 such that 


lim P 

n—>oo 


7 E (||(t[-U]|V^(G,x)> g -p^[ G;C ]||tv) >e 

11 XE V„ 


- 0 . 


To parse the above, the outer P [■] refers to the choice of G. The big {-)g is the choice of the boundary condition 
called a above. Finally, (■ |Vy(G, x))g is the random choice given the boundary condition. 

Analogously, the planted distribution ofJl has non-reconstruction\v\Xh respect to p if for any e > 0 there exists 
£ > 0 such that 


lim P 

n—* oo 


; E 

' XEV n 


<<r[-|*]|V/(G,,x)) e -p ta e { G e J 


TV/ G 


> £ 


= 0 . 


Theorem 4.5. IfM has non-reconstruction with respect to p, then is p -symmetric. If the planted distribution of 

M has non-reconstruction with respect to p, then it is p -symmetric. 


In concrete applications the non-reconstruction condition is typically reasonably easy to verify. For instance, 
in (9) we determine the precise location of the so-called “condensation phase transition” in the regular /c-SAT 
model via Theorems l4.4l and l4.5l The proof of Theorem li.SI can be found in Section [4~7l 


4.4. Gibbs uniqueness. Although the non-reconstruction condition is reasonably handy, to verify it we still need 
to “touch” the complex random graph G. Ideally, we might hope for a condition that can be stated solely in terms of 
the limiting distribution D on trees, which is conceptually far more accessible. The “Gibbs uniqueness” condition 
as put forward in j29 | fills this order. 

Specifically, suppose that T is a finite acyclic template whose root r-y is a variable node. Then we say that T is 
(e,£) -unique with respect to a marginal assignment p if 

\{(T[-\r T \\S/ ( T) t -pt\ tv <£. (4.5) 

To parse 14.51 . we observe that (a[- \rj] |Vy r) r is a random variable, namely the average of the value a[- \rj] as¬ 
signed to the root variable under the Gibbs measure pj given the values of the variables at distance at least £ from 
r-y. Hence, 14.51 requires that (a [ ■ \ r y] | Vy T) T is at total variation distance less than e for every possible assignment 
of the variables at distance at least £ from ry, i.e., for every “boundary condition”. 

More generally, we say that T Elnf is [e ,£]-unique with respect to p if the finite template d (+ 1 T has this 
property. (That d f+1 T is finite follows once more from the fact that all degrees are bounded by A.) Further, we call 
the measure D e ^(T) Gibbs-unique with respect to p if for any e > 0 we have 

lim P [T is (e ,£)-unique w.r.t. p] = 1. 


Corollary 4.6. Iff) e .^(T) is Gibbs-unique with respect to p, then 1 i m F[ln /'(G) | -9Bfj{p). 


Proof. If f) is Gibbs-unique with respect to p, then 13.91 guarantees that Ji_ has non-reconstruction with respect to 
p. Indeed, given e >(),£> 0 and a graph G let £{G, e, £) denote the set of vertices x e V n for which d f [G, x] is acyclic 
and (e ,(.) unique. Then we have 


\ E (||<ff[-l*]|V/(G > *)) 6 -p /ia #[ G>JC ]|| ) < \ £ ||||(a[-|x]|VHG,x)> G -p^ [GjX] | 

fl vr- IT ' < VJ fl V( _T J 11 

I 8{G,e,£)V 


■ xeV„ 


’ XEV n 

<£+ !-■ 


TV 


and by 13.91 P [\£{G, e,£) | < (1 - e) n] tends to 0 as n —► oo. Similarly, because the distribution of the depth-/ 7 neigh¬ 
borhood structure in the planted distribution Gy coincides with f)/ , Gibbs-uniqueness implies that the planted 
model has non-reconstruction with respect to p as well. Therefore, the assertion follows from Theorems 14.41 
and !4.5l □ 


In problems such as the random fc-SAT model, the Ising model or the Potts antiferromagnet that come with 
an "inverse temperature” parameter f > 0, Gibbs uniqueness is always satisfied for sufficiently small values of ft. 
Consequently, Corollarv l4.6l shows that the cavity method always yields the correct value of lim^oo ^E[ln Z(G)] in 
the case of small /3 , the so-called “high temperature” case in physics jargon. Furthermore, if the Gibbs uniqueness 
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condition is satisfied then there is a canonical way of constructing the marginal assignment p by means of the 
Belief Propagation algorithm (31] Chapter 14]. Hence, Co ml I a ry 14.61 p mvi d es a full comprehensive answer in this 
case. 


4.5. Meet the expectation. We proceed to prove Theorems l4.4l To this end, we need to get a handle on the condi¬ 
tional expectation of Z given ST( and for this purpose we need to study the possible empirical distributions of the 
values assigned to the variables of a concrete factor graph G e Specifically, by a ( G, ('.)-marginal sequence 

we mean a map q : ‘If —> UjL j (H- / ), T <-* qj such that 

MSI: q T e&>{Q.) if TeYn^e, 

MS2: q T e&>{D. dT ) if 

MS3: for all T e X^> n T we have 

E E ^G,AT’n{d e [T’ | j] = T}(q riJ - q T ) = 0. (4.6) 

T’£% e n£F jz[d T i] 

Thus, q assigns each tree T Ei/ rooted at a variable node a distribution on Q and each tree T eT/ rooted at a 
constraint node a distribution on Q rfr , just like in Definition l4.ll Furthermore, the consistency condition 14.61 pro¬ 
vides that for a given T rooted at a variable the average marginal distribution over all T 1 , j such that d ( [ T' ] j ] — T is 
equal to qj. However, in contrast to condition MA2 from Definition H. 1 I MS3 does not require this marginalisation 
to work out for every T', j individually. 

Suppose now that U c F n is a set of constraint nodes such that d(a) - do for all a e U. Then for a: V n —► Q we let 

a[{wi,...,w do )\U] = E U^G,a,j)) = coj)}. 

|G| aeUj =1 

Thus, cr[-|H] e SA{ Q d °) is the empirical distribution of the sequences {(cr(d(G, a, l)),...,cr(d(G, a, do))) : a e U}. A 
factor graph G and a : V n -* Q induce a (G,f)-marginal sequence qc.a/ canonically, namely the empirical distri¬ 
butions 

qG,a,e,T = 0 l-\{x£V n :d e [G,x] = T] for T e ,T e n7, 

q G ,a,e,T = o[-\{ae F n :d e+1 [G,a\ = T}] for T€.3}r&. 

Conversely, given a (G,P) -marginal sequence q let 2(G, £, q,S ) be the set of all a : V n —► Q such that for all T e T^nF, 
r'El/n^we have 

|| tfG.a.e ,t - 'frlxv - <5> || <lG,a,e,T' _ ^t'IItv - (4-7) 

Moreover, let 


Z(, q , s (G) = Z(G)(l{<jel(G,£,q,S)}) G . 


Finally, define 


I F n 


StoAt 7)= E (l-rfr)H(q r )A G ,^r|F)+^ E 

I v n\ T<= 3 - e n& 


H{q T ) + (\ny/ T {<T)) qT - D 


TeZenr 1 

In Section[5]we are going to prove the following formula for the expectation of Z(qg{G). 


(]t\\ Qs) ddF'l}]] 
j£[d T 1 


A G> ,(2m 


Proposition 4.7. For any e > 0, P > 0 there is8 > 0 such that for large enough n the following is true. Assume that 
G e ( Pl(.M n ) is 1 OOf -acyclic and let q be a (G, P)-marginal sequence. Then 

| n 1 lnE[l{^ 2 f+ 5 }^,^, 5 (G)|G =i G] - £$ G ,/(q) \ < c. 

We are going to be particularly interested in the expectation of Z(qg (G) for q “close” to a specific marginal 
assignment p. Formally, a {G,P) -marginal sequence q is (e, P) -judicious with respect to p if 


E A G /[F|F] \\q T - Pr|| TV + E E A G /[r|^] Urjy- 

Te'X/n^ je[d T ] 


TV 


< £. 


We say that (G,cr) is (£, £) -judicious with respect to p if the empirical distribution qc,a,£ is (£,^)-judicious w.r.t. p. 
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Corollary 4.8. For any a > 0 there exist e > 0, £ > 0 such that for all 0 < < e and all l>£ the following is true. Let 

££{y, l ) be the event that || Xq i - #z ||tv < T- Then 

limsup — InE [l{G £ T£{y, l) n s^ioo /}Z(G) ((G, a) is {f, l)-judicious w.r.t. p) G ] < SBoip) + a. 
rc—oo n 

Proof. Pick a small enough £ = e(a) > 0. By Lemma l4.3l there exists £ such that E[|| Pj g i T - pr || \¥\ < £for all l > £. 

Now, fix any 0 < (5,y < e, l > £, pick c, - £,((’>, l ) small enough and assume that n is big enough. Let Q(G) be the set 
of all (G, l )-marginal sequences that are (/3, Z)-judicious w.r.t. p. Because T/ is a finite set, there exists a number 
N - N(0 such that for every factor graph G there is a subset Q* (G) c Q(G) of size |Q*(G)| < N such that the 
following is true. If (G, cr) is if, /)-judicious w.r.t. p, then a e U^eq.cq L(G, l, q,0- Therefore, for all G we have 

Z(G)l{(G,cr) is (£, /)-judicious w.r.t. p} < N mM Zy^(G). (4.8) 

Pronosition l4.7l and 14.81 imply that for £ small enough and n large enough for any factor graph G e s^iooe there is 
q G e Q(G) such that 

lnE[l{^/ioo^}Z(G)(l{(G,<r) is (£,/)-judicious w.r.t. p}) G \G=( G] < 3& G j{q G ) + an/ 2. (4.9) 

To proceed, we recall that the Kullback-Leibler divergence is non-negative. Hence, 14.91 implies that for large n, 
\nE[l{srf wo c}Z(G) (1{(G,<7) is (£, Z)-judicious w.r.t. p}) G |G =( G] 

< an/2 + Y a-d T )H(q G )A G {T\y)+ l -^\ Y \ H ^ G ) + (lni^rW) A A G Om (4.10) 

TEX e nY ' V n\ L T 

Further, for any j e [A] the function v e fZfl 1 ) H (v) is uniformly continuous because , 0 Z(Q J ) is compact. By the 
same token, v>-► {]ni//(rr)) v is uniformly continuous for any i// e 'P. Consequently, if G e 5£ (y, l ) for some y < e and 
£ is chosen small enough, then 14.101 entails 

lnE[l{.«/ioo/}Z(G)(l{(G,<r) is (£,Z)-judicious w.r.t. p}) G |G=^ G] < 3Bfi{p) + an. (4.11) 

Finallt, the assertion follows from 14.111 and Bayes’ rule. □ 


Corollary 4.9. For any a > 0 there exists £ > 0 such that for all Z > £ we have 


lim P 

12—00 


-lnE[Z(C?)|5/]<flBfl(p)-o 

n 


= 0 . 


Proof. Choose a small £ = E(a) > 0. By Lemma l4~3l there exists £ such that E[|| Pigi T ~ Pr || TV < £ for all Z > £. 
Hence, fix some Z > £ and define q : T e T/ n V —<- S£(cj), T >->■ p t g i T . Moreover, for T e T; n 3F let qj e S?{jFl dT ) be 
such that H{qr) + {lm// 7 (rr)) f?j is maximum subject to the condition that q T j ; - = q g i T ^j for all j e [dj] (cf. 14.11 ). 
Further, pick S - S{e,1) > 0 small enough. Then Pronosition l4.7l imnlies that for large n and any G e .cTjoo/ 


lnE[Z/ >(?j g(G)|G —r G] > ^ G ,tlql ~ an/2 

= -an/2+ Y a-d T )Hlq T )A G j{T\y)+ l -^\ Y \ h ^t) + (]nyr T W) U G>Z (W, (4.12) 
TcX e nY TeST e n ^ 1 J 

because the definition of q ensures that the Kullback-Leibler divergences vanish. Since || f)[ - X G i ||. | v < e with high 
probability by 13.91 and E[|| p l g i T - prl^l^l < the assertion follows from 14.121 . □ 


4.6. Proof of Theorem l4.4l We begin by spelling out the following consequence of the symmetry assumption. 


Lemma 4.10. If.J/ is p -symmetric, then for any e > 0 for all sufficiently large £ wehave 
ll/tGJjc — P(,d c [G,x] II TV > £n 


lim P 

12—*00 


xeVn 


lim P 

72—oo 


X ^G/i* Pe,d e lG f ,x] TV >£ ” 

X£V n " 


= lim P [ p G fails to be (£, 2) - symmetric 1 = 0 and 
12—00 1 1 

= lim P Up fails to be (£, 2) -symmetric] = 0. 

n —»rYi 1 ^ f 


(4.13) 

(4.14) 
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Proof. Choose 77 = 77(G) > 0 small enough. For an integer £ > 0 consider the event 

8 t = \ E I m !*,y! - IG,X) ® P/,3* [G,y] || Tu < ^ « 2 

UyeV n " " iV 

If is /'.(-symmetric, then lim^,*, P [G e 6f\ for sufficiently large £. Similarly, if the planted distribution is p- 
symmetric, then limbecP \G( e 8(\ for large £. 

Hence, assume that GeS(. Then by the triangle inequality, for any 01 e Q, 


7 E \Pt,d‘lG.x](°>)-»G\x(0>)\ = -\ E 


n XEV n 

Therefore, 


x<LV n 


E E Pe,a ( \G,x\ (m) P( : gi [G,y] fa') 
yeV„w'en 


E E MGUy(w.w') 

yuVnw'zn 


^ 2 . 


E l|p^[G,x]-mx|| T v^ 7 ? 2 i°i <7 7- 


Furthermore, by 14.151 and the triangle inequality, 


r „ || || 

— E WPGIx® PGiy-Pe,ge{G,x]® P(,d f {G,y]\\ TV - 2r l- 


x,yeV n 


(4.15) 


(4.16) 


Since G £ <§/, 14.161 entails that 


~2 E \\PGlx®PGly-PGHx,y}\\ TV ^3ri<e, 

n x,yeV„ 


i.e., Gis (e, 2 )-symmetric. 


□ 


Lemma 4.11. There is a number £0 = £0 (A, 0,^,0) such that for all 0 < e < £ 0 , £ > 0 there exists % > 0 such that for 
large enough n the following is true. IfG £ ,: S(.J£ n ) is a (2£ + 5) -acyclic factor graph such that 

E \\PGix-Pe, d t [ G,x]\\y V <£3n (4 - 17) 

xaV n 

and pc is (j, 2) -symmetric, then (l{(G,<r) is (e,£)-judicious w.r.t. p}) G > 1/2. 

Proof. Pick S - S(£,e) > 0 small, (’> - (HS) and y = jiff smaller and % - y(y) > 0 smaller still and assume that 
n > n o(y). Let Vo be the partition of V n such that x,y e V n belong to the same class iff d' +2 [G, x| = d (+2 [G,y]. 
By Theorem [2T| there exists a refinement V of Vo such that pc is y-homogeneous with respect to (V, S) for some 
partition S of O” such that #V + #S < N - N(y). We may index the classes of V as Vtj with T = d e+2 [G,x] for all x 
in the class and i £ [Nj] for some integer Nj. 

Let / be the set of all j £ [#S] such that piSj) >6 7 IN and p[-\Sj] is y-regular. Then by HM1 

E/t(S ; -)> l-<5 6 . (4.18) 

jzJ 

Further, Lemma [2781 shows that Sj is a (/i,2)-state if j e J. Therefore, choosing % small enough, we obtain from 
Corollarv l2.4l that 


- E \\PGixl-\S j ]-pGix\\ TV <8 7 for all jej. 

n X£V n 

Therefore, by 14.171 and the triangle inequality, for j e J we get 

7 ; E \\PGixi\Sj] - e 3 + 1- £ ||p GU [-|S i ]-pGi.v|| xv <£ 3 + 35 7 <2e 3 . 

n xeVn n X£Vn 
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Consequently, by 14.181 . Bayes’ rule and the triangle inequality, summing over all T e X/ +2 n Y and i e [Np] we get 

-LlVr,il<||<r[-|Vr,<]-p/ I r|| T v> G = -L E I V r>i (|| <r[ -1 V r ,,] - P^rM^c 

n T,i n T,i ;e[#S] 

<<S 7 + -£ E l^r,ilMG(S;) ||<ff[ - |Vr,/]|Sy) G - p^rlTv [byHM2] 

n T,ija[#S] 

* s7 + tX E E Vg(Sj) ||/iGJ*[-|Sy] - Petfmx] llwOe 3 - (4.19) 

“ T,i xeV T ,i jel#Si 


Applying the triangle inequality once more, we find 

E ^G,e[T\Y] (|| qG,a,e,T~ P(,t\\tv) - 

Tn% e cy 


-Z\ v T.i\(\\<rl-\VT,i]-P(,T 

n T,i 


IITV/G 


<3e 3 


(4.20) 


Further, consider T EX;nf such that Xc.t I T\&\ > 0 and let j e [dp]. Because G is [21 + 5)-acyclic, there exists 
a set T{T,j) c X/ +2 n V with the following two properties. First, for every constraint node a with d r+> [G, a] — T the 
variable node x - d[G, a,j ) satisfies d f+2 [G, x] eT[T,j). Second, for every variable node x with d e+2 [G,x] e T[T,j) 
there is a constraint node a with d c+l [G, a] = T such that d[G,a,j) = x. For ReT[T,j) let niRjj be the number 
of constraint nodes a with d t+l [G, a] = T such that x - d[G,a,j) satisfies d f+2 [G, x] = R. Then by the triangle 
inequality, 


E E *G,ATm ( 1 qG,a,e,T\j - PitUT]]] L) r 

TeZ t n&Md T ] Miv/G 

5 I E I ^(k[.|^i- w ,lw> c 

reX/n^ye[d r ]fier(rj) 

^7 X E (lkMVH,/]-p^|| T v) G ; 

n R€%( +2 ny ie[d R ] 


(4.21) 


the last inequality follows because all degrees are between one and A. Finally, the assertion follows from 14.191 , 

E2ID and E2D. □ 


We proceed by proving the upper bound and the lower bound statement from Theorem 53] separately. Strictly 
speaking, the proof of the lower bound implies the upper bound as well. But presenting the arguments separately 
makes them slightly easier to follow. 

Proof of Theorem [4pH upper bound. Fore,/> 0 let <?(£,/) = {Z.xEV n Ipgi* - Pi,d l [G,x] IItv < Additionally, let S£[%) 
be the event that po is (j, 2)-symmetric and let 5£[e, /) be the event that || Ag,/ - || TV < £• We assume that Jl_ is 
p- symmetric. 

Given a > 0 choose a small enough e > 0 and a large enough £ > 0 as promised by Corollarv l4.8l By Lemma 14731 
there is £* > £ such that 

E [l|P/,a'r _ Pr|| TV l 7/ ] <£4 for all Z>^*. (4.22) 

Let x = X(£,^») be the number provided by Lemma l4.11l Then Lemma l4.10l implies that lim^ooP [G e SP[x)\ = 1. 
Similarly, Lemma l4.10l implies that for large enough / we have lim„^oo P [G e <?(£ 4 , l )] = 1 Hence, the local conver¬ 
gence assumption 0.91 implies that for all large enough /, 

lim P [G e S£[y) n if (e 4 , /) n <?(£ 4 , /)] = 1. (4.23) 

n—*oo 

Further, we claim that f £(£ 4 , /) r\S[e 4 , l) c 5£(e 4 ,£*) n(?(£ 3 ,^*)- Indeed, if / > £„, then ££{e 4 ,l) c 5£[e 4 ,£*). More¬ 
over, if G £ (e 4 , /) n S[e 4 , l), then with xeV n chosen uniformly at random we find 

E | Pg\x - Pe, } gf, [G,x] | xy - E II PG\x - P /,31 [G,x] II tv + E || ,d t * [G,*] ~~ Pl,d l [G,*] || TV 

<e 4 + Y. ^mlUr-p^.rlL 

TeXjnr " " 1V 

< £ 4 + 21| 0, [• \Y] - A G>/ [• \Y] IItv + E [ || Pl # T - p,^. T \Y\ < £ 3 . 


22 
















Consequently, combining 14.221 and 14.231 , we find that the event n i? ( £ 4 , ^ *) n <? ( £ 3 , ^ *) satisifes 

lim P [G £ S&[a)] = 1. (4.24) 

n—* oo 

Further, if G £ £&{a) n .^ioot,, then Z(G) < 2Z(G) (l{(G,<r) is ( £ ,/*)-judicious w.r.t. p}) G by Lemma 14.111 and the 
choice of X- Therefore, 

E[1{G £ 5^(a) n^/ioo/, }Z(G)] < 2E [1 {Ge ,S?( £ 4 ,^„) n.s/ioo £ „}(l{(G,<7) is ( £ ,/„)-judicious w.r.t. p}) G Z(G )]. (4.25) 
Since > f, for large enough n Corollarv l4.8l and 14.251 yield 

E[liG£^(a)n^ 100 /,}Z(G)] < 2exp(rc(5§ fl (p) + a)). (4.26) 

Further, combining 14.241 and 14.381 and using Markov’s inequality, we conclude that 

lim P [Z(G) > exp[n{3Sf){p) + 2a))\3S{a)nsrfiooe,] = 0. 

72—-CO 

Therefore, 14.241 . the high girth assumption and Proposition s.2l vield 

lim P [Z(G) > exp (n(SSg(p) + 2a))l = 0. (4.27) 

72—00 1 1 

Finally, since |n _1 lnZ(G)[ is bounded by some number C = C(A,Q,'P,0) > 0 by the definition 13.41 of Z, 14.271 
implies that lim sup n -1 E[lnZ(G)] < SS^ip) + 3 a. Taking a —► 0 completes the proof. □ 

To establish the lower bound we introduce a construction reminiscent of those used in I l20l [2T1 127. [341 [39l . 
Namely, starting from the sequence M_ of (A, Q, 'F, 0) -models, we define another sequence M ® - (,M ®) „ of mod¬ 
els as follows. Let Q® = fl x Q and let us denote pairs £ Q® by (» * co'. Further, for any y/ : fl h —>• (0,oo) we 

define a function 

y/® : (Q®) ,! — (0,oo), {(Oi®(o' v ...,a)h®(o' h )~y/(a>i,...,a) h )-y/{(o' v ...,(o' h ). 

Let 'P® = {yr® : yr e 'Ll. Then the (A,Q.'P,©)-model M n = (V n ,F n ,d n , t n , (y/ a ) a <iP n ) gives rise to the (A,Q®, v F®,0)- 
model Jf® - ( V „, F n , d, t, [y/®) aeFn ). 

Clearly, there is a canonical bijection ( S [,M) —«• ( S(.M®), G >-► G®. Moreover, the construction ensures that the 
Gibbs measure pc® e& > {fl® n ) equals pc® Pg- Explicitly, for aRu>i,o)' v ...,o) n ,w' n Efl, 

Pc® (oil ® <j/|. (O n ®a)' n ) = pQ{(Oi,...,W n )pG{(o' v ...,(o'n). (4.28) 

In effect, we obtain 

Z(G®) = Z(G) 2 . (4.29) 

Further, writing ©*,T® for the (A,Q®, 'P®,0)-templates and the acyclic (A, fl®, 'P®,0)-templates, we can lift the 
marginal assignment p from ‘X to T® by letting p® 8 = pj ® pr for all T. Additionally, let fl® e ^(X®) be the image 
of fl under the map T £ X —► T® £ X® so that 

»fl.(p®)=2# fl (p). (4.30) 

Proof of Theorem \4l\ lower bound. We assume that .M_ is p-symmetric and that the same is true of the planted 
distribution. For e, / > 0 consider the event 

^®( £ ,Z) Ji £ |b»u- P VX<4- (431) 

[ XEVn ) 

and let £P®{x) be the event that p G « is (j, 2)-symmetric. Moreover, as before we let f£{e,() - {|| \q ( - i)f || |v < e\. 
Basically, we are going to apply the same argument as in the proof of the upper bound to the random factor graph 
G® and to G( for a large enough f:. 

Flence, let a > 0. Then Corol 1 a rv l4.8l annlied to .M ® yields a small e = e(a) > 0 and a large £ — £ (a) > 0. Moreover, 
Corollarv l4.9l nrovides a large £'{a) > 0. Further, by Lemma POI and 14.281 there exists £*> £ + (' such that 

E[|b^r-Pr|| TV l^]+E[||p® (? , r8 -p® (8 || Tv |F]< £ 4 forall*>*.. (4.32) 

Applying Lemma l4.11l to M ®, we obtain %* - X* (G^*) > 0 and Proposition ^.5l and l.emma 11.101 imp 1 y that 

lim P[G£^®(x*)1 = 1. (4.33) 

72—00 
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Further, Lemma 14.101 shows that for Z we have 


lim P [G e <?®(£ 4 , Z)] = 1. (4.34) 

22-00 1 1 

In effect, just as before 14.33b 14.341 and 13.91 show that large Z, 

lim P [G e £?®(x*) n 5£[£ 4 , Z) n (e 4 ,2)1 - 1. (4.35) 

22—00 

Like in the upper bound proof we have J£(£ 4 , l)n<§®{£ 4 , Z) ci?(f 4 , Z)n<?®(£ 3 ,^,)- Therefore, 14.321 and 14.351 show 
that the event 3§®{a) - £f®lx*) (M£{£ 4 ,£*) C\S®[£ 3 ,£*) satisfies 

lim P[Ge^®(a)] = 1. (4.36) 

22—00 L J 


Define ,2T a (G) = 1{G e £$®(a) n ^/ioo/»}^(G). If G e £$®(a) n ^/ioof,> then by 14.29b Lemma l4.11l and the choice 
of x* we have 

Z(G) 2 = Z(G®) < 2Z(G®) (l{(G®,o-) is (e,<?*)-judicious w.r.t. p®}) G<9 . 

Hence, we obtain an upper bound on the second moment of ,Z a , namely 

E[Za(G) 2 ] <2E[1{G Eif(£ 4 ,f,)n ^100/,}(1{(G®,0-) is (e,^-judicious w.r.t. p®}) G ® Z(G®)]. (4.37) 

Due to 14.301 and the choice of £, (. and because £* > £, Corollary 14.81 enables us to estimate the r.h.s. of 14.371 
explicitly, whence 

E[ Z e [G) 2 ] < exp(n(2^ fl (p) + a )). (4.38) 

As a next step, we are going to show that 

E[J? £ (G)] > exp {n{S&${p®) - 2a)). (4.39) 

Indeed, by Proposition s.5l and l.emma fl. 1 Ol ive have 

lim P [G; e «Z’®(l»)] = 1 (4.40) 

22-00 

for large enough Z. Similarly, 14.28b the assumption that the planted distribution is p-symmetric and Lemma l4.10l 
imply that for Z large enough 

lim P [G/ £ <a®(£ 4 ,2) 1 = 1. (4.41) 

22-00 1 

Hence, 14.401 , 14.411 , the local convergence assumption 13.91 and the construction 13.101 of the planted distribution 
imply that for Z large enough 

lim P[G;£^®(x»)n^f(£ 4 ,Z)n<?®(£ 4 ,Z)] = 1. (4.42) 

Combining 14.321 and 14.421 and using the high girth assumption, we thus obtain for large Z 

lim P [G; e S&® (a)l = 1. (4.43) 

22-00 L 1 

Further, Corollarv l4.9l shows that 


lim P 

22-00 


-lnE[Z(G)|5/] > 3$e(p) - alstiooi 


= 1 . 


Thus, 14.431 and Pronosition l3.8l vield 14.391 . 

Finally, combining 14.381 and 14.391 and applying the Paley-Zygmund inequality, we obtain 

pr 'yT 2 

P[Z(G) > exp(n(& e (p) - 4a))] >P[2 £ (G) > exp(rc(33 fl (p)-4a))] > * >exp(-10aw). 

Zh[~z. £ [{j) \ 

As this holds for any a > 0, the assertion follows from 14.441 and Proposition s. 21 


(4.44) 

□ 
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4.7. Proof ofTheorem l4.51 The key step of the proof is to establish the following statement. 

Lemma 4.12. For any e > 0 there exists 8 > 0 such that for any £ > 0 there exists no such that for all n> no the 
following is true. Assume that G e satisfies 

7 E (||<*H*]|V/(G,x)> M -p mGiJe] || ) <S 9 . (4.45) 

n x<lV„ e 

Then G is (e,2) -symmetric and Y.xev„ \\pg\x - Pe,d e [G,x) || TV < £n ' 

Before we prove Lemma l4.12l let us show how it implies Theorem l4.5l 

Proof of Theorem frEl If G e ( §{,M n ) satisfies is (e, 2)-symmetric and T. X ev n \\pgix - P(,d e [G,x] II tv < en ’ ^ len by the 
triangle inequality 

E WG\{x,y}~ Pe,^[G,x] ®P(,d f lG,y] L, - E II PG\{x,y] - PG\x ® PG\y ||tv + MGJx ® PG)y ~ Pe^t[G,x] ® P(,d e [G,y] L, 

x,yeV„ 1V x,yeV„ 

< 4 en 2 . 

Therefore, the theorem follows by applying Lemma [4.121 either to the random factor graph G or to the random 
factor graph G( chosen from the planted model. □ 

Proof of Lemma [4J2[ Let y = y(e) > 0 be sufficiently small. By Theorem l2.1l we can pick 8 - 8 (y) > 0 small enough 
so that there exists a partition (V, S) with #V+#S < d _1 with respect to which p G is y 4 -homogeneous. Suppose that 
Vi, Sj are classes such that | V;| > S 3l2 n, pc(Sj) > d 3/2 and such that p [ • |Sj | is y 4 -regular on V). We claim that 

7777 E II MGJxI' IS ; -] - ||-re < 3y• (4.46) 

I m'I xeVj 

The assertion is immediate from this inequality. Indeed, suppose that 14.461 is true for all i, j such that \V, \ > 8 312 n, 
PG(Sj) > 8 312 such that p[- \Sj] is y 4 -regular on V,. Then because #V + #S <1/8 

E \\PGixl-\Sj]-p^ d f[ G ,x]\\ TV <^rn. (4.47) 

X€V n 

Hence, by HM1 and Bayes’ rule, T.xev„ \\pgix - Pe d'\G x] IItv < '^ n < £n ‘ I“ L1 rtheer, 14.471 and Lemma l2l)l implv that 
pc is (e, 2) -regular (provided that we pick y small enough). Thus, we are left to prove 14.461 . 

Assume for contradiction that 14.461 is violated for Vi, Sj such that \Vi\ > 8 3l2 n, pc,{Sj) > 8 312 . Then by the 
triangle inequality there is a set W c Vi of size at least y|V,| such that for all x e W we have 

\\PGixi-\Sj]-Pe^ [G:X] 1-jY- 7- 

For xeW pick oi x e 11 such that \pcix\w x \Sj\ ~ P(g([ Gx ]\-Y i s maximum. Then by the pigeonhole principle there 
exist a) e FI and W’ c W, \ W’\ > |W|/(2|n|), such that either 

Vx e W’ : hg[ x [ a )| Sj] > p ed e [GiX] («) + y or 

Vx g W ’: n Glx [w\Sj] < Pc }d e [GtX] («) - y 

In particular we have 

Vxe W': fi Glx [w\Sj] > p £d e [Gx] l(o) + yl\n\ 

We claim that there is a set L c W' of size \L\ = (1 18] with the following properties. 

(i) the pairwise distance between any two x,y eL is at least I 0(7 + 1). 

(ii) for all x e L we have 

(||(o'[-[x]|V/(G,x)>g - Pe, d t lGiX] || w ) < d 4 . 

Indeed, because | V)| > 8 2 n and p(S ; ) > d 2 the assumption 14.451 implies that 

E ( 1 (O’[• U]|(G, X)> G - p^ a q G>;c] || w ) <d 5 |V,|. 

xeV( 'MGL-Isp 

Since \ W'\ > y|Vi|/|Q| > 8\Vi\, 14.521 implies that there is a set W" e W' of size \ W"\ > | W'|/2 such that 14.511 holds 
for all x e W". Now, construct a sequence W" - W ( " W" ■ ■ ■ inductively as follows. In step i > 1 pick some x,- e 


(4.48) 

(4.49) 

(4.50) 

(4.51) 

(4.52) 
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W"_ !. Then W" contains x; and all y e W"_ { \ 1 x; } whose distance from x ( - is greater than 10(7 + 1). Since for each x; 
the total number of variable nodes at distance at most 10(7+1) is bounded by A 10t/+1) and \W^ \ > d|V/|/2 > S 3 nl 2, 
the set n*>i W. 1 has size at least 5 3 A _10 ^ +1) n/2 > 1/d, provided that n is large enough. Finally, simply pick any 
subset Lc f|/>i W" of size \L\ = [1/dl. 

Consider the event <? = {tr[<y|L] > |L| _1 Y. X aL Pe a e [G x] + 7 3 } ■ We claim that 

p G [g\Sj]<2S 2 . (4.53) 

Indeed, by 14.511 and the union bound we have 

(l{v*eL:||<a[-|x]|V^(G,*)> G -p /ia / [G ,1 <fi}) > 1- ^ (l{||<o-[-|x]|VHG,x)> G -p w#|GA . | || TV >d}) 

>l-<5 2 . (4.54) 

Now, let £ be the coarsest u-algebra such that £ => V^(G, x) for all x e L. Suppose that ct e Sj is such that 

|| <cr[- \x]\SI C (G,x)> g (ct) - p ea e [Gx] < 6 for all xeL. 

We claim that 14.551 implies 

(l{a £ <f}|£> G (ct) < 5 3 . 

Indeed, let X — Y. X ai, l{c(x) = to}. Then 14.551 implies that 

<X(<7)|£> (ct) < 25\L\ + £ p eid e [G:X] {to). (4.57) 

xeL 

Furthermore, the pairwise distance of the variables in L is at least 2(7 + 1) and given £ the values of the variables 
at distance either 7 or £ + 1 from each x e L are fixed. Therefore, given £ the events {ct(x) = a>\ are mutually 
independent. In effect, X is stochastically dominated by a sum of independent random variables. Hence, recalling 
that S is much smaller than y, we see that 14.561 follows from 14.571 and the Chernoff bound. Finally, combining 
14.541 and 14.561 we obtain 14.531 . 

But 14.531 does not sit well with 14.501 . In fact, 14.501 entails that /i G [<fjSy] > y 2 ; for consider the random variable 
Y = Ex£L 1{ct(x) ^ w}. Then 1450} yields < Xxe/.O-bclxMSy]) < |L|(l-y/|0|)-£^ £ i Pe,d e [G,x] ( w )- Hence - 

by Markov’s inequality 

l _ _ <*Vis,] _„ \L\a-rim-ZxeLPe, d e [G ,x]^ < i-r/iQi ^ 2 

1 ~ \L\{l-y 3 )-'£ xliL p ei ge lG ' X] {(0) - \L \(1 - y 3 ) - ExeL Pe,d e [ G ,x] ( w ) ~~ 1 ~T 3 ~ Y ' 

Combining this bound with 14.531 . we obtain y 2 < p G (rS’)//i G (S ; ) < 2 S 2 I p G (Sfi. Thus, choosing S much smaller 
than y, we conclude that p G (Sj) < S ’’ 12 , which is a contradiction. Thus, we have established that 14.461 . □ 

5. Conditioning on the local structure 

5.1. A generalised configuration model. The aim in this section is to prove Proposition [477] The obvious prob¬ 
lem is the conditioning on the u-algebra 3~( that fixes the depth-7 neighborhoods of all variable nodes and the 
depth-7 + 1 neighborhoods of all constraint nodes. Following |[14|, we deal with this conditioning by setting up a 
generalised configuration model. 

Recall that is the (finite) set of all isomorphism classes d { T for T e Xn V and d^ +1 T for T e Tn Y. Let £,n> 0 
be integers and let Jl - ( V,F,d , t,(y/ a )a^F ) be a (A,Q, v P,Q)-model of size n. Moreover, let G e be a 1007- 

acyclic factor graph. Then we define an enhanced (A,n,'F,Q/)-model ,M{G,£) with type set 0/ = (T/ n Y) x [A] 
as follows. The set of variable nodes is V, the set of constraint nodes is F, the degrees are given by d and the 
weight function associated with each constraint a is y/ a just as in ,M. Moreover, the type of a variable clone (x, i) 
is f G /(x,i) = {d ( [G,x],i). Further, the type of a constraint clone ( a,j ) such that d{G,a,j ) = (x,/) is t G: ({a,j) = 
{d ( [G,x],i). Clearly, ( d(.M(G,£)) c ( £j(.M ). The following lemma shows that the model .M(G,£) can be used to 
generate factor graphs whose local structure coincides with that of G. 

Lemma 5.1. Assume that 7 > 0 and that G' e '£1 (.M (G, 7)) is 27 + 4 -acyclic. Then G' viewed as a .M -factor graph 
satisfies G=e G'. 


(4.55) 

(4.56) 
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Proof. We are going to show inductively for l e [£] that G=i G'. The case / = 0 is immediate from the construction. 
Thus, assume that / > 0, let (x, i) e CV and let B be the set of all clones that have distance precisely l - 1 from (x, i). 
Since G' is (2( + 2)-acyclic, the pairwise distance of any two clones in B is at least 2. Moreover, by induction we 
know that t Gi \ ( w > j) - to ,t (w,j) for all ( w, j) e B. Therefore, t G q (x, i) - t G qi (x, i). □ 

In order to prove Proposition l4.7l we need to enhance the model .M{G,£) further to accommodate an assignment 
that provides a value from D for each clone. Thus, let o : CV U Cf —► O be a map. We call & valid if <r(x, i) - <J(x, j) 
for all x e V, i, j e \d(x)] and if for all 0 e 0/ we have 

Mix) e Q : |{(x, i) e Cy : b(x, i) = id, to,A x, i) = 0}| = \{(a,j) e Cf : &{a,j) = o, t Gi Aa,j) = 0}|. 

Of course, we can extend a valid & to a map V —> Cl, X 1 -* 6{x, 1). Given a valid <t we define a model (A, Cl, V P, 0/ x Q) - 
model J£[G,d,£) with variable nodes V, constraint nodes F, degrees d and weight functions ( ys a )a£F such that 
the type to,a,Ax, i ) of a variable clone (x, i) is {d e [G,x], i,i f (x, /)) and such that the type t f ; ,>/(«,/) of a constraint 
clone ( a,j ) with d{G,a,j ) = (x,i) is (cK[ G,x],i,d{a, j)). By construction, !,£)) c ^{JTIGA)) c 

Let us recall the definition of the distance from 0.51 . Further, for two maps &,&' : Cy U Cf —«■ Q let dist(<7,<7') = 

| {O, i) e Cy U Cf : a( v, i) f d'(v, /))} |. In Section [hl2l we are going to establish the following. 

Lemma 5.2. For any e,£ > 0 there is no - no(£,£,A,Cl, V T,0) such that for n > hq the following holds. If M is a 
(A,Cl,'¥,(-))-model of size n, G e ,: £1(.M) is 100 £-acyclic and a is valid, then with probability at least 1 - c the random 
factor graph G{Jl[G,o ,£f) has the following property. There exist a valid &' and a A £-acyclic G' e ( d(.M(G,<T',(:)) 
such that dist(d,&) + dist[G', G{JF{G,d,£))) < n °- 9 . 

To proceed consider a (G, ^-marginal sequence if. We call a q-validif the following two conditions hold. 

VI: For all T e X^ n Y,io e Cl we have 

||xe V:d e [G,x] = T,ct(x) = w|| = < 7 r(w)|{x£ V:/[G,x] = jj|. 

V2: For all T e X^ n a>i,..., io £ Cl we have 

|| aaF:d ul [G,a] = T,Mj e [d F ]: &{a,j) = w ; -}| = q T ((o l ,...,(o dT ) ||a e F: d e+1 [G, a] = r}|. 

Lemma 5.3. Foranye,£ > 0 thereisno = no{e,£,A,Cl, V P,0) such thatforn > no the following holds. Assume that M 
is a (A, Q, T 1 ,0 )-model of size n, G e 7? {.M) is 1 00 £-acyclic and q is a (G, £)-marginal sequence such that there exists 
a if-valid a. Then with the sum ranging over all q-valid o we have 

exp [nS$ G {£, q) - Vn) < £ \cS{M(G £))\ ~ 6XP + ^ • 

We defer the proof of Lemma liOl to Section [A3l 
Proof of Provosition \4 . 71 We claim that 

|{G' e : G' = e G}| > m^(G,£))\ exp(-n 091 ). (5.1) 

To see this, apply Lemma [A2l to the constant map a : (v, j) e Cy u Cf co o for some fixed a>o e Cl. Then we conclude 

that with probability at least 1/2 the random graph G{Jl[G,£y) = G{M[G,a,£f) is at distance at most n 09 from 
a 4/'-acyclic G' e G(.J/(G,£)) c ( fi(.M). Furthermore, by Lemma I5TT1 this factor graph G', viewed as an element of 
(.M), satisfies G = t G'. Finally, since the total number of factor graphs at distance at most n° LJ from G' is bounded 
by exp(n 0 91 ) because all degrees are bounded, we obtain 15. 1L 

Let S > 0 be small enough. If cr e Y.{G,£,q,8), then by 1 14.71 there exists a [G ,£]-marginal sequence q' such 
that a e 1,{G,£, q', 0) such that || — <7^-1| TV < <5 for all T e %(. Because X/ is finite and I.[G,£, q', 0) f 0, the total 

number of such q' is bounded by a polynomial in n. Moreover, due to the continuity of Ac, A d we can choose 
8 = 8{£) small enough so that \S£q : A<i') - S&G.Aq )I < £/2 for all such q'. Hence, summing over all a corresponding 
to cr e l.[G,£, q,8), we obtain from 15. II and Lemma IX3l that 

E[Z^ 0 (G)|G=^G] <V-!-<exp [n3B G {£,q) + £n). 

' q ^ |{G' e ^{Min )): G' =e G}| F ' 

Conversely, by Lemma [5721 with probability at least 1/2 the graph Gj(.M(G,n,(. )) is within distance at most n 0 9 of 
a 4^-acyclic G', which satisfies G' G by Lemma [Q1 As before, the total number of graphs at distance at most 
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n 0 9 off G' is bounded by exp ( n os1 ). Similarly, the total number of a' at distance at most ?i 0 9 off <x is bounded by 
exp(n 0 91 ). Therefore, bvLemma l5.ll 


E[l{s4 2U1 }Z ( , q [G)\G= e G]> 


exp(-2n 098 ) 

2 


E 


|^M7(G,<7,7))| 
YS(M(G, 7))| 


> exp(n5§G(7, £7) - £ri), 


as desired. 


□ 


5.2. Proof of Lemma [572l Let 0, = { tc,d,e (x, i ): (x, i) e Cy} be the set of all possible types. For each re0, let n r be 
the number of clones (x, i) e Cy with t Gil y,e (x, i) = r. Throughout this section we assume that n> no (e, 7, A, Q, 'L, 0) 
is sufficiently large. 

Lemma 5.4. There exists (’> > 0 such that the following is true. For any G,a there exists 3/4 < y < 7/8 such that for 
every r e 0* either n r < n r orn T > n Y+ ^. 

Proof. The number of possible types is bounded independently of n. Hence, choosing p small enough, we can 
ensure that there exists an integer j > 0 such that 3/4+ j p <718 such that [ n 3l4+ W, n 3li+ ^ +1 ^]n{n T : t e T} - 0. □ 

Fix f, y as in the previous lemma. Call t rare if n T < n Y and common otherwise. Let Y be the number of variable 
clones that belong to cycles of length at most 107 in G(,J/(G, a, (.)). 

Lemma 5.5. For large enough n we have E[F] < n Y In n. 

Proof. Let R be the set of variable clones {v,i) of a rare type and let U be the set of all variable clones whose distance 
from R in G does not exceed 1007. Since the maximum degree as well as the total number of types are bounded, we 
have \ U\ < R \In In n < n Y \/\nn, provided that n is big enough. Thus, to get the desired bound on E[F] we merely 
need to consider the set W of common clones that are at distance more than 1007 from R. 

More specifically, let (t/, i) be a common clone. We are going to bound the probability that (v, i ) e IV and that 
(v, i) lies on a cycle of length at most 107. To this end, we are going to explore the (random) factor graph from 
(v, i) via the principle of deferred decisions. Let i\ = i,...,ii e [A] be a sequence of l < 107 indices. If ( v, i) lies on 
a cycle of length at most 107, then there exists such a sequence {i\,...,ip that corresponds to this cycle. Namely, 
with v\ - v the cycle comprises of the clones (iq, (iq, ip such that d(GM7(G,cr,7)), Vj, ij) = {Vj+ 1 , ij+i). In 

particular, vi — V\ . Clearly, the total number of sequences (i\ . ip is bounded. Furthermore, given that (i//, ip is 

common, the probability that vi - vq is bounded by 2n~ Y . Since y > 3/4, the linearity of expectation implies that 
E[T] < |C/| + 2n l ~7 Inn< n Y Inn. □ 

Lemma 5.6. Assume that G" e (G, d, 7)) satisfies Y (G") < n Y In 2 n. Then there is a 4 7 -acyclic G' e ( T1 {.M (G, 7)) 
such that dist[G',G") < n 0 9 . 

Proof. Let R be the set of variable clones (v, i ) of a rare type and let U be the set of all variable clones whose 
distance from R in G does not exceed 107. Moreover, let G"' e ( fl(.M(G,t)) minimise dist(G",G"') subject to the 
condition that d(G"', v, i) - d(G, v, i ) for all ( v, i) e U. Then dist(G", G'") < n ! ' In n because the total number of types 
is bounded. Therefore, the assumption Y (G") < n Y In 2 n implies that Y (G'") < n Y In 3 n, say. In addition, because G 
is 1007-acyclic, none of the clones in R lies on a cycle of length at most 47 in G'". 

Altering only a bounded number of edges in each step, we are now going to remove the short cycles of G"' one 
by one. Let C be the set of common clones. The construction of G'" ensures that only common clones lie on cycles 
of length at most 47. Consider one such clone ( v, i) and let N be the set of all variable clones that can be reached 
from ( v , i) by traversing precisely two edges of G'"; thus, N contains all clones ( w , j) such that w has distance two 
from v and all clones (v, j) that are incident to the same constraint node as (v, i). Once more by the construction 
of G'" we have N c C. Furthermore, |AT| < A 2 . 

We claim that there exists N 1 c C and a bijection ^: N —>• N 1 such that the following conditions are satisfied. 

(i) t G ,a,e ( w, j ) = tc.ijj (<C( w, jP for all ( w, j ) e N. 

(ii) the pairwise distance in G'" between any two clones in N' is at least 1007. 

(iii) the distance in G'" between Nu{(v, /)} and N' is at least 1007. 

(iv) the distance between R and N' is at least 1007. 

(v) any {w,j) e N' is at distance at least 1007 from any clone that belongs to a cycle of G'" of length at most 47. 
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Since the maximum degree of G'" is bounded by A, there are no more than n Y In 4 n clones violate condition (iii), 
(iv) or (v). By comparison, there are at least n Y+ l’ clones of any common type. Hence, the existence of £ follows. 
Now, obtain G"" from G'" as follows. 

• let G""U;(w,j)) = G'"[w,j ) and G""{w,j ) = G'"tf(w,j)) for all (w,j) e N. 

• let G"" ( w, j) = G'" ( w,j ) for all (w } j) gNuN'. 

It is immediate from the construction that any clone on a cycle of length at most At in G"" also lies on such a cycle 
of G'". Moreover, {v, i) does not lie on a cycle of length at most At in G"". Hence, Y [G"") < Y (G'"). In addition, all 
clones on cycles of length at most At and their neighbours are common. Hence, the construction can be repeated 
on G"". Since Y(G"') < n r In 3 n, we ultimately obtain a At -acyclic G" with dist(G',G") < « >'ln 4 n < n 03 . □ 

Proof of Lemmci\K2\ The assertion is immediate from I .e m m as 1 5.5 1 a n d [5. 6 1 a n d Markov’s inequality. □ 


5.3. Proof of Lemma T5.31 Let Tp — %( nV and for T e Tf let n-p be the number of variable nodes x such that 
d f [G, x] — T. By Stirling’s formula the number |Z(G, t, q, 0) | of assignments o : V„ —* Q with marginals as prescribed 
by q satisfies 


< In n. 


(5.2) 


ln|Z| - Y n T H(q T ) 

TaYf 

Further, for T e Yp and i e [dp] let Cy{T, i) be the set of all clones (x, i) e Cy such that t ( -;y{x, i) - (7’, i). Moreover, 
let Cp(T, i) be the set of all clones (a, j) e Cp such that tcj- ia, j) = (T, i). Additionally, let YFf ( T, i) be the set of all 
pairs [T', j) with TeJ/H 9>, j e [dpi] such that there is (a, j) e Cp{T, i) such that d f+1 [G, a} - T'. Of course, the 
total number of perfect matchings between C\/ ( T, i ) and Cp{T, i ) equals rip\. If we fixer e Z(G, t, q, 0), then any such 
perfect matching induces an assignment & : Cp ( T, i) —* O by mapping a clone ( a,j ) e Cp(T, i ) matched to (x, i) to 
the value ct(x). Let B- r j be the event that in a such random matching for all ( T',j ) e dtfiT, i ) and all o> we have 

\{(a,j)eC P :d M [G,a] = T',cna,j)=CD}\ = q r \j (w) 11 (a,j)eC F : d {+1 [G, a] = 2 v j | 

Moreover, for [T',j) £ &({T, i ) let m T i be the number of a e F such that d e+1 [G, a] = T'. Then 


P IB,] = — 
n t \ 


n 

men 


qT{a>)nT 

v (</rqy(w)mr')(r'j)e^(r,o 


n 


m T ' 


[qr\j(oS)m T ')\ 

{T',j)^f{T,i),a)en 


nj 

(qr{oj)n.T)bjen) 


\ _1 

n 

{T' 


rriji 


exp 


0(ln n) - Y m r D{q V[ j\\q T ) 

VC ,j)n& t {T,i) 


Let - ‘Xf n dt. Multiplying up over all ( T, i), we obtain for B = fl Bpi 


P[B]= n n P [•Br.j] = exp 

i£[dj] 


0(ln n ) - E E m T'D\qT'\jWqai[T'\j]\ 

T'a&e jE[d T i] 


(5.3) 


where the constant hidden in the O(-) depends on A,f), 'P.O, t only. 

Further, for T' e dtp let Sp be the event that for every .,a>d ,) £ we have 


H 


ae F :d 


/+! 


Then 


P[S r \B] = 


171T 1 Qt 1 

Hence, letting S = flSj-', we obtain 


[G, a\ = T\ Vj £ [d r ] :&la,j) = a)j}\ = q r bo 1 ,...,w dr , ) | {a e F: d ui [G, a] = T }|. 

= exp [Q(ln n) - m T :D[q r II qr \i ® ■ ■ • ® <?rq d T ,)] • 


fflf 


n 

jtldfi] 


mpi 

\mrq T ’i] t 


P [S|B] = exp 


O(lnn) - Y m T'DlqrWqT'\i®---® qrid T ,) 


T'e&t 


Once more the constant hidden in the 0(•) depends on A,Q,'T,0, t only. Further, given SnB we have 


n Vaicr) = exp 

aeF 


Y m T' (lnt^rdo - )) 
T'a&i 


At' 


(5.4) 


(5.5) 
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Finally, the assertion follows from I l5.2l - f531 . 
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